Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1613 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1182. Min Range: -4.1182

Max Act: 4.1182. Min Act: -0.1700

Data Index: 3129944 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 4.1182. Min Range: -4.1182

Max Act: 4.0918. Min Act: -0.1700

Data Index: 643596 (Open Web Text)

Max Activating Token Index: 2

Text #2

Max Range: 4.1182. Min Range: -4.1182

Max Act: 4.0857. Min Act: -0.1700

Data Index: 6620880 (Open Web Text)

Max Activating Token Index: 10

Text #3

Max Range: 4.1182. Min Range: -4.1182

Max Act: 4.0741. Min Act: -0.1700

Data Index: 5212200 (Open Web Text)

Max Activating Token Index: 216

Text #4

Max Range: 4.1182. Min Range: -4.1182

Max Act: 4.0923. Min Act: -0.1700

Data Index: 100640 (Open Web Text)

Max Activating Token Index: 869

Text #5

Max Range: 4.1182. Min Range: -4.1182

Max Act: 4.0804. Min Act: -0.1700

Data Index: 6865753 (Open Web Text)

Max Activating Token Index: 2

Text #6

Max Range: 4.1182. Min Range: -4.1182

Max Act: 4.0513. Min Act: -0.1700

Data Index: 3204125 (Open Web Text)

Max Activating Token Index: 353

Text #7

Max Range: 4.1182. Min Range: -4.1182

Max Act: 4.0211. Min Act: -0.1700

Data Index: 4204776 (Open Web Text)

Max Activating Token Index: 166

Text #8

Max Range: 4.1182. Min Range: -4.1182

Max Act: 4.0335. Min Act: -0.1700

Data Index: 7203930 (Open Web Text)

Max Activating Token Index: 2

Text #9

Max Range: 4.1182. Min Range: -4.1182

Max Act: 4.0335. Min Act: -0.1700

Data Index: 2272627 (Open Web Text)

Max Activating Token Index: 2

Text #10

Max Range: 4.1182. Min Range: -4.1182

Max Act: 4.0335. Min Act: -0.1700

Data Index: 433604 (Open Web Text)

Max Activating Token Index: 2

Text #11

Max Range: 4.1182. Min Range: -4.1182

Max Act: 3.9837. Min Act: -0.1700

Data Index: 3709818 (Open Web Text)

Max Activating Token Index: 376

Text #12

Max Range: 4.1182. Min Range: -4.1182

Max Act: 3.9742. Min Act: -0.1700

Data Index: 8059682 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 4.1182. Min Range: -4.1182

Max Act: 4.0070. Min Act: -0.1700

Data Index: 6725331 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 4.1182. Min Range: -4.1182

Max Act: 3.9643. Min Act: -0.1700

Data Index: 6482312 (Open Web Text)

Max Activating Token Index: 59

Text #15

Max Range: 4.1182. Min Range: -4.1182

Max Act: 3.9796. Min Act: -0.1700

Data Index: 165337 (Open Web Text)

Max Activating Token Index: 13

Text #16

Max Range: 4.1182. Min Range: -4.1182

Max Act: 3.9828. Min Act: -0.1700

Data Index: 909426 (Open Web Text)

Max Activating Token Index: 581

Text #17

Max Range: 4.1182. Min Range: -4.1182

Max Act: 3.9759. Min Act: -0.1700

Data Index: 948770 (Open Web Text)

Max Activating Token Index: 446

Text #18

Max Range: 4.1182. Min Range: -4.1182

Max Act: 3.9617. Min Act: -0.1700

Data Index: 663669 (Open Web Text)

Max Activating Token Index: 531

Text #19

Max Range: 4.1182. Min Range: -4.1182

Max Act: 3.9485. Min Act: -0.1700

Data Index: 2903930 (Open Web Text)

Max Activating Token Index: 67