Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 392 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.8843. Min Act: -0.1700

Data Index: 7555533 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.8546. Min Act: -0.1700

Data Index: 7555532 (Open Web Text)

Max Activating Token Index: 73

Text #2

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.8562. Min Act: -0.1700

Data Index: 4158611 (Open Web Text)

Max Activating Token Index: 8

Text #3

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.8033. Min Act: -0.1700

Data Index: 1326364 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7600. Min Act: -0.1700

Data Index: 8436612 (Open Web Text)

Max Activating Token Index: 1020

Text #5

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7412. Min Act: -0.1700

Data Index: 1633315 (Open Web Text)

Max Activating Token Index: 9

Text #6

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7337. Min Act: -0.1700

Data Index: 2213619 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7437. Min Act: -0.1700

Data Index: 3420600 (Open Web Text)

Max Activating Token Index: 7

Text #8

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7373. Min Act: -0.1700

Data Index: 2273938 (Open Web Text)

Max Activating Token Index: 14

Text #9

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7395. Min Act: -0.1700

Data Index: 4302802 (Open Web Text)

Max Activating Token Index: 678

Text #10

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7582. Min Act: -0.1700

Data Index: 650572 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7587. Min Act: -0.1700

Data Index: 3278386 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7530. Min Act: -0.1700

Data Index: 8303413 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7429. Min Act: -0.1700

Data Index: 2973070 (Open Web Text)

Max Activating Token Index: 152

Text #14

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7033. Min Act: -0.1700

Data Index: 1843916 (Open Web Text)

Max Activating Token Index: 6

Text #15

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7286. Min Act: -0.1700

Data Index: 6120078 (Open Web Text)

Max Activating Token Index: 5

Text #16

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7068. Min Act: -0.1700

Data Index: 6181827 (Open Web Text)

Max Activating Token Index: 15

Text #17

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7366. Min Act: -0.1700

Data Index: 6415941 (Open Web Text)

Max Activating Token Index: 5

Text #18

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7205. Min Act: -0.1700

Data Index: 2788730 (Open Web Text)

Max Activating Token Index: 991

Text #19

Max Range: 3.8843. Min Range: -3.8843

Max Act: 3.7145. Min Act: -0.1700

Data Index: 6791707 (Open Web Text)

Max Activating Token Index: 45