Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2335 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0211. Min Range: -4.0211

Max Act: 4.0211. Min Act: -0.1700

Data Index: 8569851 (Open Web Text)

Max Activating Token Index: 37

Text #1

Max Range: 4.0211. Min Range: -4.0211

Max Act: 4.0016. Min Act: -0.1700

Data Index: 8439490 (Open Web Text)

Max Activating Token Index: 535

Text #2

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.9378. Min Act: -0.1700

Data Index: 4082175 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.8735. Min Act: -0.1700

Data Index: 8156762 (Open Web Text)

Max Activating Token Index: 723

Text #4

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7865. Min Act: -0.1700

Data Index: 2707880 (Open Web Text)

Max Activating Token Index: 2

Text #5

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7689. Min Act: -0.1700

Data Index: 916432 (Open Web Text)

Max Activating Token Index: 43

Text #6

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7945. Min Act: -0.1700

Data Index: 40315 (Open Web Text)

Max Activating Token Index: 896

Text #7

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7837. Min Act: -0.1700

Data Index: 8356712 (Open Web Text)

Max Activating Token Index: 469

Text #8

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7277. Min Act: -0.1700

Data Index: 1749565 (Open Web Text)

Max Activating Token Index: 154

Text #9

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7274. Min Act: -0.1700

Data Index: 2380540 (Open Web Text)

Max Activating Token Index: 935

Text #10

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7709. Min Act: -0.1700

Data Index: 761491 (Open Web Text)

Max Activating Token Index: 398

Text #11

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7412. Min Act: -0.1700

Data Index: 423649 (Open Web Text)

Max Activating Token Index: 122

Text #12

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7753. Min Act: -0.1700

Data Index: 3999215 (Open Web Text)

Max Activating Token Index: 71

Text #13

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7329. Min Act: -0.1700

Data Index: 8165092 (Open Web Text)

Max Activating Token Index: 8

Text #14

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7330. Min Act: -0.1700

Data Index: 2214652 (Open Web Text)

Max Activating Token Index: 634

Text #15

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7139. Min Act: -0.1700

Data Index: 6424679 (Open Web Text)

Max Activating Token Index: 1011

Text #16

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7767. Min Act: -0.1700

Data Index: 7413602 (Open Web Text)

Max Activating Token Index: 393

Text #17

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7167. Min Act: -0.1700

Data Index: 6955217 (Open Web Text)

Max Activating Token Index: 5

Text #18

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7358. Min Act: -0.1700

Data Index: 7176847 (Open Web Text)

Max Activating Token Index: 68

Text #19

Max Range: 4.0211. Min Range: -4.0211

Max Act: 3.7359. Min Act: -0.1700

Data Index: 4809465 (Open Web Text)

Max Activating Token Index: 801