Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2870 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.3184. Min Act: -0.1700

Data Index: 180911 (Open Web Text)

Max Activating Token Index: 818

Text #1

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.2504. Min Act: -0.1700

Data Index: 506803 (Open Web Text)

Max Activating Token Index: 377

Text #2

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.1896. Min Act: -0.1700

Data Index: 4233745 (Open Web Text)

Max Activating Token Index: 153

Text #3

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.1256. Min Act: -0.1700

Data Index: 5567437 (Open Web Text)

Max Activating Token Index: 29

Text #4

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.1325. Min Act: -0.1700

Data Index: 5503622 (Open Web Text)

Max Activating Token Index: 343

Text #5

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.1458. Min Act: -0.1700

Data Index: 6269827 (Open Web Text)

Max Activating Token Index: 834

Text #6

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.0911. Min Act: -0.1700

Data Index: 7875503 (Open Web Text)

Max Activating Token Index: 489

Text #7

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.0950. Min Act: -0.1700

Data Index: 7431463 (Open Web Text)

Max Activating Token Index: 105

Text #8

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.0836. Min Act: -0.1700

Data Index: 7901858 (Open Web Text)

Max Activating Token Index: 307

Text #9

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.0596. Min Act: -0.1700

Data Index: 7836912 (Open Web Text)

Max Activating Token Index: 941

Text #10

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.0708. Min Act: -0.1700

Data Index: 460504 (Open Web Text)

Max Activating Token Index: 709

Text #11

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.0559. Min Act: -0.1700

Data Index: 901902 (Open Web Text)

Max Activating Token Index: 275

Text #12

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.0323. Min Act: -0.1700

Data Index: 3648398 (Open Web Text)

Max Activating Token Index: 692

Text #13

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.0389. Min Act: -0.1700

Data Index: 2923536 (Open Web Text)

Max Activating Token Index: 590

Text #14

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.0196. Min Act: -0.1700

Data Index: 262459 (Open Web Text)

Max Activating Token Index: 281

Text #15

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.0097. Min Act: -0.1700

Data Index: 1666337 (Open Web Text)

Max Activating Token Index: 940

Text #16

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.0103. Min Act: -0.1700

Data Index: 6941487 (Open Web Text)

Max Activating Token Index: 601

Text #17

Max Range: 3.3184. Min Range: -3.3184

Max Act: 2.9794. Min Act: -0.1700

Data Index: 7314771 (Open Web Text)

Max Activating Token Index: 77

Text #18

Max Range: 3.3184. Min Range: -3.3184

Max Act: 2.9976. Min Act: -0.1700

Data Index: 4313726 (Open Web Text)

Max Activating Token Index: 226

Text #19

Max Range: 3.3184. Min Range: -3.3184

Max Act: 3.0025. Min Act: -0.1700

Data Index: 5357137 (Open Web Text)

Max Activating Token Index: 75