Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2482 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.6477. Min Act: -0.1700

Data Index: 2800275 (Open Web Text)

Max Activating Token Index: 48

Text #1

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.6269. Min Act: -0.1700

Data Index: 245333 (Open Web Text)

Max Activating Token Index: 595

Text #2

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.5999. Min Act: -0.1700

Data Index: 4301352 (Open Web Text)

Max Activating Token Index: 716

Text #3

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.6004. Min Act: -0.1700

Data Index: 6274818 (Open Web Text)

Max Activating Token Index: 587

Text #4

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.5580. Min Act: -0.1700

Data Index: 1766381 (Open Web Text)

Max Activating Token Index: 920

Text #5

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.5716. Min Act: -0.1700

Data Index: 399296 (Open Web Text)

Max Activating Token Index: 427

Text #6

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.5132. Min Act: -0.1700

Data Index: 2549005 (Open Web Text)

Max Activating Token Index: 272

Text #7

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.5362. Min Act: -0.1700

Data Index: 3172211 (Open Web Text)

Max Activating Token Index: 439

Text #8

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.5478. Min Act: -0.1700

Data Index: 5763591 (Open Web Text)

Max Activating Token Index: 124

Text #9

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.5321. Min Act: -0.1700

Data Index: 5516751 (Open Web Text)

Max Activating Token Index: 67

Text #10

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.4962. Min Act: -0.1700

Data Index: 3384531 (Open Web Text)

Max Activating Token Index: 249

Text #11

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.5027. Min Act: -0.1700

Data Index: 3403242 (Open Web Text)

Max Activating Token Index: 78

Text #12

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.4243. Min Act: -0.1700

Data Index: 7950288 (Open Web Text)

Max Activating Token Index: 761

Text #13

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.4477. Min Act: -0.1700

Data Index: 6859129 (Open Web Text)

Max Activating Token Index: 48

Text #14

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.4599. Min Act: -0.1700

Data Index: 6129509 (Open Web Text)

Max Activating Token Index: 438

Text #15

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.4507. Min Act: -0.1700

Data Index: 1967127 (Open Web Text)

Max Activating Token Index: 108

Text #16

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.4478. Min Act: -0.1700

Data Index: 5237704 (Open Web Text)

Max Activating Token Index: 285

Text #17

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.4357. Min Act: -0.1700

Data Index: 2036410 (Open Web Text)

Max Activating Token Index: 513

Text #18

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.4376. Min Act: -0.1700

Data Index: 7365983 (Open Web Text)

Max Activating Token Index: 786

Text #19

Max Range: 3.6477. Min Range: -3.6477

Max Act: 3.4685. Min Act: -0.1700

Data Index: 4381433 (Open Web Text)

Max Activating Token Index: 462