Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 26 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.5377. Min Act: -0.1700

Data Index: 4916494 (Open Web Text)

Max Activating Token Index: 149

Text #1

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.5146. Min Act: -0.1700

Data Index: 6745992 (Open Web Text)

Max Activating Token Index: 615

Text #2

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.4740. Min Act: -0.1700

Data Index: 1144885 (Open Web Text)

Max Activating Token Index: 907

Text #3

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.4374. Min Act: -0.1700

Data Index: 3700398 (Open Web Text)

Max Activating Token Index: 25

Text #4

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.4018. Min Act: -0.1700

Data Index: 3448211 (Open Web Text)

Max Activating Token Index: 237

Text #5

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3860. Min Act: -0.1700

Data Index: 8352782 (Open Web Text)

Max Activating Token Index: 174

Text #6

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3843. Min Act: -0.1700

Data Index: 1645601 (Open Web Text)

Max Activating Token Index: 559

Text #7

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3873. Min Act: -0.1700

Data Index: 1703728 (Open Web Text)

Max Activating Token Index: 757

Text #8

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.4160. Min Act: -0.1700

Data Index: 6184733 (Open Web Text)

Max Activating Token Index: 266

Text #9

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3919. Min Act: -0.1700

Data Index: 3758187 (Open Web Text)

Max Activating Token Index: 277

Text #10

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3733. Min Act: -0.1700

Data Index: 3199879 (Open Web Text)

Max Activating Token Index: 26

Text #11

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3488. Min Act: -0.1700

Data Index: 2276188 (Open Web Text)

Max Activating Token Index: 542

Text #12

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3730. Min Act: -0.1700

Data Index: 2359258 (Open Web Text)

Max Activating Token Index: 484

Text #13

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3695. Min Act: -0.1700

Data Index: 2797495 (Open Web Text)

Max Activating Token Index: 987

Text #14

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3612. Min Act: -0.1700

Data Index: 808980 (Open Web Text)

Max Activating Token Index: 85

Text #15

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3434. Min Act: -0.1700

Data Index: 6755781 (Open Web Text)

Max Activating Token Index: 928

Text #16

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3267. Min Act: -0.1700

Data Index: 6839940 (Open Web Text)

Max Activating Token Index: 995

Text #17

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3231. Min Act: -0.1700

Data Index: 7014824 (Open Web Text)

Max Activating Token Index: 571

Text #18

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3203. Min Act: -0.1700

Data Index: 5857387 (Open Web Text)

Max Activating Token Index: 171

Text #19

Max Range: 3.5377. Min Range: -3.5377

Max Act: 3.3420. Min Act: -0.1700

Data Index: 1339969 (Open Web Text)

Max Activating Token Index: 719