Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1031 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.6481. Min Act: -0.1700

Data Index: 7414551 (Open Web Text)

Max Activating Token Index: 714

Text #1

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.5551. Min Act: -0.1700

Data Index: 1331164 (Open Web Text)

Max Activating Token Index: 958

Text #2

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.5524. Min Act: -0.1700

Data Index: 998241 (Open Web Text)

Max Activating Token Index: 1018

Text #3

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.5662. Min Act: -0.1700

Data Index: 1952571 (Open Web Text)

Max Activating Token Index: 496

Text #4

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.5532. Min Act: -0.1700

Data Index: 998242 (Open Web Text)

Max Activating Token Index: 191

Text #5

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.5269. Min Act: -0.1700

Data Index: 4398164 (Open Web Text)

Max Activating Token Index: 459

Text #6

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.5334. Min Act: -0.1700

Data Index: 3338869 (Open Web Text)

Max Activating Token Index: 564

Text #7

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.5199. Min Act: -0.1700

Data Index: 4694759 (Open Web Text)

Max Activating Token Index: 484

Text #8

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.5275. Min Act: -0.1700

Data Index: 7615735 (Open Web Text)

Max Activating Token Index: 568

Text #9

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.4952. Min Act: -0.1700

Data Index: 5208483 (Open Web Text)

Max Activating Token Index: 439

Text #10

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.5056. Min Act: -0.1700

Data Index: 4123895 (Open Web Text)

Max Activating Token Index: 77

Text #11

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.4974. Min Act: -0.1700

Data Index: 3255024 (Open Web Text)

Max Activating Token Index: 464

Text #12

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.4866. Min Act: -0.1700

Data Index: 5945603 (Open Web Text)

Max Activating Token Index: 558

Text #13

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.4837. Min Act: -0.1700

Data Index: 7009781 (Open Web Text)

Max Activating Token Index: 217

Text #14

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.4728. Min Act: -0.1700

Data Index: 3889447 (Open Web Text)

Max Activating Token Index: 49

Text #15

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.4813. Min Act: -0.1700

Data Index: 92300 (Open Web Text)

Max Activating Token Index: 878

Text #16

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.4697. Min Act: -0.1700

Data Index: 3831201 (Open Web Text)

Max Activating Token Index: 676

Text #17

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.4665. Min Act: -0.1700

Data Index: 3171498 (Open Web Text)

Max Activating Token Index: 176

Text #18

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.4744. Min Act: -0.1700

Data Index: 1114369 (Open Web Text)

Max Activating Token Index: 515

Text #19

Max Range: 1.6481. Min Range: -1.6481

Max Act: 1.4742. Min Act: -0.1700

Data Index: 2661968 (Open Web Text)

Max Activating Token Index: 758