Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2575 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.6712. Min Act: -0.1700

Data Index: 2999548 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.6383. Min Act: -0.1700

Data Index: 6564203 (Open Web Text)

Max Activating Token Index: 12

Text #2

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.6440. Min Act: -0.1700

Data Index: 2712109 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.6228. Min Act: -0.1700

Data Index: 4331877 (Open Web Text)

Max Activating Token Index: 10

Text #4

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.6087. Min Act: -0.1700

Data Index: 249907 (Open Web Text)

Max Activating Token Index: 912

Text #5

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.6110. Min Act: -0.1700

Data Index: 3746290 (Open Web Text)

Max Activating Token Index: 12

Text #6

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.6037. Min Act: -0.1700

Data Index: 7269444 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5877. Min Act: -0.1700

Data Index: 4121305 (Open Web Text)

Max Activating Token Index: 688

Text #8

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5958. Min Act: -0.1700

Data Index: 2004115 (Open Web Text)

Max Activating Token Index: 870

Text #9

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5888. Min Act: -0.1700

Data Index: 6243321 (Open Web Text)

Max Activating Token Index: 88

Text #10

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5884. Min Act: -0.1700

Data Index: 2723554 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5917. Min Act: -0.1700

Data Index: 200468 (Open Web Text)

Max Activating Token Index: 9

Text #12

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5939. Min Act: -0.1700

Data Index: 3436166 (Open Web Text)

Max Activating Token Index: 67

Text #13

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5885. Min Act: -0.1700

Data Index: 6253565 (Open Web Text)

Max Activating Token Index: 37

Text #14

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5732. Min Act: -0.1700

Data Index: 2115852 (Open Web Text)

Max Activating Token Index: 2

Text #15

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5732. Min Act: -0.1700

Data Index: 4407894 (Open Web Text)

Max Activating Token Index: 2

Text #16

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5732. Min Act: -0.1700

Data Index: 4628891 (Open Web Text)

Max Activating Token Index: 2

Text #17

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5782. Min Act: -0.1700

Data Index: 492073 (Open Web Text)

Max Activating Token Index: 691

Text #18

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5703. Min Act: -0.1700

Data Index: 3968500 (Open Web Text)

Max Activating Token Index: 18

Text #19

Max Range: 2.6712. Min Range: -2.6712

Max Act: 2.5812. Min Act: -0.1700

Data Index: 3872087 (Open Web Text)

Max Activating Token Index: 6