Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1428 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1944. Min Range: -5.1944

Max Act: 5.1944. Min Act: -0.1700

Data Index: 7440320 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 5.1944. Min Range: -5.1944

Max Act: 5.1056. Min Act: -0.1700

Data Index: 5925219 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 5.1944. Min Range: -5.1944

Max Act: 5.0752. Min Act: -0.1700

Data Index: 7315382 (Open Web Text)

Max Activating Token Index: 2

Text #3

Max Range: 5.1944. Min Range: -5.1944

Max Act: 5.0822. Min Act: -0.1700

Data Index: 8001153 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 5.1944. Min Range: -5.1944

Max Act: 5.0567. Min Act: -0.1700

Data Index: 8767911 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 5.1944. Min Range: -5.1944

Max Act: 5.0091. Min Act: -0.1700

Data Index: 3398204 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 5.1944. Min Range: -5.1944

Max Act: 5.0071. Min Act: -0.1700

Data Index: 4996388 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 5.1944. Min Range: -5.1944

Max Act: 5.0254. Min Act: -0.1700

Data Index: 5655345 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 5.1944. Min Range: -5.1944

Max Act: 5.0166. Min Act: -0.1700

Data Index: 5300285 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 5.1944. Min Range: -5.1944

Max Act: 5.0077. Min Act: -0.1700

Data Index: 7407517 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 5.1944. Min Range: -5.1944

Max Act: 5.0078. Min Act: -0.1700

Data Index: 4351502 (Open Web Text)

Max Activating Token Index: 3

Text #11

Max Range: 5.1944. Min Range: -5.1944

Max Act: 4.9874. Min Act: -0.1700

Data Index: 4277855 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 5.1944. Min Range: -5.1944

Max Act: 4.9986. Min Act: -0.1700

Data Index: 1648678 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 5.1944. Min Range: -5.1944

Max Act: 4.9947. Min Act: -0.1700

Data Index: 1766602 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 5.1944. Min Range: -5.1944

Max Act: 5.0030. Min Act: -0.1700

Data Index: 7305709 (Open Web Text)

Max Activating Token Index: 3

Text #15

Max Range: 5.1944. Min Range: -5.1944

Max Act: 4.9597. Min Act: -0.1700

Data Index: 4456254 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 5.1944. Min Range: -5.1944

Max Act: 4.9498. Min Act: -0.1700

Data Index: 4715613 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 5.1944. Min Range: -5.1944

Max Act: 4.9600. Min Act: -0.1700

Data Index: 91106 (Open Web Text)

Max Activating Token Index: 3

Text #18

Max Range: 5.1944. Min Range: -5.1944

Max Act: 4.9560. Min Act: -0.1700

Data Index: 6480036 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 5.1944. Min Range: -5.1944

Max Act: 4.9600. Min Act: -0.1700

Data Index: 4972566 (Open Web Text)

Max Activating Token Index: 3