Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 883 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.7015. Min Act: -0.1700

Data Index: 2625894 (Open Web Text)

Max Activating Token Index: 620

Text #1

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.6215. Min Act: -0.1700

Data Index: 971714 (Open Web Text)

Max Activating Token Index: 135

Text #2

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.6631. Min Act: -0.1700

Data Index: 7772997 (Open Web Text)

Max Activating Token Index: 551

Text #3

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.6278. Min Act: -0.1700

Data Index: 1961535 (Open Web Text)

Max Activating Token Index: 91

Text #4

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.5587. Min Act: -0.1700

Data Index: 6515421 (Open Web Text)

Max Activating Token Index: 553

Text #5

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.6041. Min Act: -0.1700

Data Index: 4858393 (Open Web Text)

Max Activating Token Index: 140

Text #6

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.5629. Min Act: -0.1700

Data Index: 436241 (Open Web Text)

Max Activating Token Index: 369

Text #7

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.5105. Min Act: -0.1700

Data Index: 3540100 (Open Web Text)

Max Activating Token Index: 310

Text #8

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.5261. Min Act: -0.1700

Data Index: 567460 (Open Web Text)

Max Activating Token Index: 626

Text #9

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.4962. Min Act: -0.1700

Data Index: 6653980 (Open Web Text)

Max Activating Token Index: 561

Text #10

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.5066. Min Act: -0.1700

Data Index: 1657447 (Open Web Text)

Max Activating Token Index: 219

Text #11

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.4598. Min Act: -0.1700

Data Index: 3972412 (Open Web Text)

Max Activating Token Index: 575

Text #12

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.4786. Min Act: -0.1700

Data Index: 6767699 (Open Web Text)

Max Activating Token Index: 112

Text #13

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.4511. Min Act: -0.1700

Data Index: 1000691 (Open Web Text)

Max Activating Token Index: 208

Text #14

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.4558. Min Act: -0.1700

Data Index: 1777884 (Open Web Text)

Max Activating Token Index: 218

Text #15

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.4307. Min Act: -0.1700

Data Index: 4326942 (Open Web Text)

Max Activating Token Index: 538

Text #16

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.4470. Min Act: -0.1700

Data Index: 3909874 (Open Web Text)

Max Activating Token Index: 356

Text #17

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.4778. Min Act: -0.1700

Data Index: 7863459 (Open Web Text)

Max Activating Token Index: 380

Text #18

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.4688. Min Act: -0.1700

Data Index: 8137678 (Open Web Text)

Max Activating Token Index: 458

Text #19

Max Range: 3.7015. Min Range: -3.7015

Max Act: 3.4144. Min Act: -0.1700

Data Index: 5135655 (Open Web Text)

Max Activating Token Index: 457