Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2925 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3957. Min Act: -0.1542

Data Index: 4849015 (Open Web Text)

Max Activating Token Index: 1013

Text #1

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3675. Min Act: -0.1471

Data Index: 7922614 (Open Web Text)

Max Activating Token Index: 657

Text #2

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3518. Min Act: -0.0714

Data Index: 7536993 (Open Web Text)

Max Activating Token Index: 635

Text #3

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3667. Min Act: -0.1236

Data Index: 8778534 (Open Web Text)

Max Activating Token Index: 979

Text #4

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3491. Min Act: -0.1012

Data Index: 5084050 (Open Web Text)

Max Activating Token Index: 872

Text #5

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3346. Min Act: -0.1132

Data Index: 8193238 (Open Web Text)

Max Activating Token Index: 988

Text #6

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3428. Min Act: -0.1170

Data Index: 1733986 (Open Web Text)

Max Activating Token Index: 825

Text #7

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3484. Min Act: -0.1687

Data Index: 3291348 (Open Web Text)

Max Activating Token Index: 962

Text #8

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3447. Min Act: -0.1404

Data Index: 7223536 (Open Web Text)

Max Activating Token Index: 969

Text #9

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3382. Min Act: -0.0813

Data Index: 961801 (Open Web Text)

Max Activating Token Index: 996

Text #10

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3434. Min Act: -0.0575

Data Index: 1696706 (Open Web Text)

Max Activating Token Index: 583

Text #11

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3350. Min Act: -0.1058

Data Index: 2511193 (Open Web Text)

Max Activating Token Index: 1014

Text #12

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3236. Min Act: -0.1517

Data Index: 7297262 (Open Web Text)

Max Activating Token Index: 676

Text #13

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3301. Min Act: -0.0280

Data Index: 7439611 (Open Web Text)

Max Activating Token Index: 94

Text #14

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3257. Min Act: -0.0968

Data Index: 2966578 (Open Web Text)

Max Activating Token Index: 1018

Text #15

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3376. Min Act: -0.1094

Data Index: 775719 (Open Web Text)

Max Activating Token Index: 528

Text #16

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3191. Min Act: -0.0601

Data Index: 4212983 (Open Web Text)

Max Activating Token Index: 998

Text #17

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3220. Min Act: -0.0633

Data Index: 2354396 (Open Web Text)

Max Activating Token Index: 599

Text #18

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3110. Min Act: -0.1278

Data Index: 8142675 (Open Web Text)

Max Activating Token Index: 974

Text #19

Max Range: 1.3957. Min Range: -1.3957

Max Act: 1.3199. Min Act: -0.0510

Data Index: 7513804 (Open Web Text)

Max Activating Token Index: 646