Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 905 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0947. Min Range: -5.0947

Max Act: 5.0947. Min Act: -0.1700

Data Index: 616355 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 5.0947. Min Range: -5.0947

Max Act: 5.0554. Min Act: -0.1700

Data Index: 4043067 (Open Web Text)

Max Activating Token Index: 8

Text #2

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9956. Min Act: -0.1700

Data Index: 6822628 (Open Web Text)

Max Activating Token Index: 12

Text #3

Max Range: 5.0947. Min Range: -5.0947

Max Act: 5.0326. Min Act: -0.1700

Data Index: 1120413 (Open Web Text)

Max Activating Token Index: 838

Text #4

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9354. Min Act: -0.1700

Data Index: 4522566 (Open Web Text)

Max Activating Token Index: 19

Text #5

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9535. Min Act: -0.1700

Data Index: 4543860 (Open Web Text)

Max Activating Token Index: 34

Text #6

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9457. Min Act: -0.1700

Data Index: 5033562 (Open Web Text)

Max Activating Token Index: 14

Text #7

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9444. Min Act: -0.1700

Data Index: 5281527 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9474. Min Act: -0.1700

Data Index: 998066 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9287. Min Act: -0.1700

Data Index: 5955627 (Open Web Text)

Max Activating Token Index: 4

Text #10

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9397. Min Act: -0.1700

Data Index: 5691542 (Open Web Text)

Max Activating Token Index: 255

Text #11

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9045. Min Act: -0.1700

Data Index: 3113274 (Open Web Text)

Max Activating Token Index: 940

Text #12

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9216. Min Act: -0.1700

Data Index: 6735746 (Open Web Text)

Max Activating Token Index: 12

Text #13

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9197. Min Act: -0.1700

Data Index: 519667 (Open Web Text)

Max Activating Token Index: 18

Text #14

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9148. Min Act: -0.1700

Data Index: 5805205 (Open Web Text)

Max Activating Token Index: 377

Text #15

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9062. Min Act: -0.1700

Data Index: 5498850 (Open Web Text)

Max Activating Token Index: 8

Text #16

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9059. Min Act: -0.1700

Data Index: 472918 (Open Web Text)

Max Activating Token Index: 784

Text #17

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.9232. Min Act: -0.1700

Data Index: 732047 (Open Web Text)

Max Activating Token Index: 13

Text #18

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.8929. Min Act: -0.1700

Data Index: 2575127 (Open Web Text)

Max Activating Token Index: 10

Text #19

Max Range: 5.0947. Min Range: -5.0947

Max Act: 4.8882. Min Act: -0.1700

Data Index: 3176474 (Open Web Text)

Max Activating Token Index: 397