Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2643 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.4104. Min Act: -0.1700

Data Index: 1348357 (Open Web Text)

Max Activating Token Index: 945

Text #1

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.3937. Min Act: -0.1700

Data Index: 8032105 (Open Web Text)

Max Activating Token Index: 991

Text #2

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.3765. Min Act: -0.1700

Data Index: 7537403 (Open Web Text)

Max Activating Token Index: 897

Text #3

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.3916. Min Act: -0.1700

Data Index: 7265821 (Open Web Text)

Max Activating Token Index: 186

Text #4

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.3455. Min Act: -0.1700

Data Index: 3245166 (Open Web Text)

Max Activating Token Index: 205

Text #5

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.3200. Min Act: -0.1700

Data Index: 7843312 (Open Web Text)

Max Activating Token Index: 628

Text #6

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.3206. Min Act: -0.1700

Data Index: 7086563 (Open Web Text)

Max Activating Token Index: 2

Text #7

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.2794. Min Act: -0.1700

Data Index: 6150593 (Open Web Text)

Max Activating Token Index: 941

Text #8

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.2964. Min Act: -0.1700

Data Index: 7307580 (Open Web Text)

Max Activating Token Index: 911

Text #9

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.2689. Min Act: -0.1700

Data Index: 5222066 (Open Web Text)

Max Activating Token Index: 362

Text #10

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.2776. Min Act: -0.1700

Data Index: 7796532 (Open Web Text)

Max Activating Token Index: 921

Text #11

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.2814. Min Act: -0.1700

Data Index: 2809815 (Open Web Text)

Max Activating Token Index: 859

Text #12

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.2869. Min Act: -0.1700

Data Index: 5171999 (Open Web Text)

Max Activating Token Index: 837

Text #13

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.2542. Min Act: -0.1700

Data Index: 5905506 (Open Web Text)

Max Activating Token Index: 720

Text #14

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.2739. Min Act: -0.1700

Data Index: 7779640 (Open Web Text)

Max Activating Token Index: 376

Text #15

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.2123. Min Act: -0.1700

Data Index: 4628340 (Open Web Text)

Max Activating Token Index: 1021

Text #16

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.2394. Min Act: -0.1700

Data Index: 7225630 (Open Web Text)

Max Activating Token Index: 745

Text #17

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.1909. Min Act: -0.1700

Data Index: 2852639 (Open Web Text)

Max Activating Token Index: 398

Text #18

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.2298. Min Act: -0.1700

Data Index: 2243954 (Open Web Text)

Max Activating Token Index: 171

Text #19

Max Range: 4.4104. Min Range: -4.4104

Max Act: 4.2284. Min Act: -0.1700

Data Index: 5731925 (Open Web Text)

Max Activating Token Index: 517