Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1006 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.5323. Min Act: -0.1700

Data Index: 3337145 (Open Web Text)

Max Activating Token Index: 48

Text #1

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.4550. Min Act: -0.1700

Data Index: 8573407 (Open Web Text)

Max Activating Token Index: 436

Text #2

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.4434. Min Act: -0.1700

Data Index: 6363557 (Open Web Text)

Max Activating Token Index: 78

Text #3

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.3922. Min Act: -0.1700

Data Index: 3972543 (Open Web Text)

Max Activating Token Index: 277

Text #4

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.3656. Min Act: -0.1700

Data Index: 7480150 (Open Web Text)

Max Activating Token Index: 65

Text #5

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.3472. Min Act: -0.1700

Data Index: 7158264 (Open Web Text)

Max Activating Token Index: 417

Text #6

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.3736. Min Act: -0.1700

Data Index: 8437386 (Open Web Text)

Max Activating Token Index: 941

Text #7

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.3480. Min Act: -0.1700

Data Index: 8097538 (Open Web Text)

Max Activating Token Index: 549

Text #8

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.2869. Min Act: -0.1700

Data Index: 4975816 (Open Web Text)

Max Activating Token Index: 71

Text #9

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.2691. Min Act: -0.1700

Data Index: 4192731 (Open Web Text)

Max Activating Token Index: 358

Text #10

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.2413. Min Act: -0.1700

Data Index: 4243139 (Open Web Text)

Max Activating Token Index: 715

Text #11

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.2542. Min Act: -0.1700

Data Index: 414027 (Open Web Text)

Max Activating Token Index: 717

Text #12

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.2331. Min Act: -0.1700

Data Index: 2964700 (Open Web Text)

Max Activating Token Index: 814

Text #13

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.2149. Min Act: -0.1700

Data Index: 2878927 (Open Web Text)

Max Activating Token Index: 281

Text #14

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.2402. Min Act: -0.1700

Data Index: 680439 (Open Web Text)

Max Activating Token Index: 568

Text #15

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.2241. Min Act: -0.1700

Data Index: 6812412 (Open Web Text)

Max Activating Token Index: 443

Text #16

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.2276. Min Act: -0.1700

Data Index: 690440 (Open Web Text)

Max Activating Token Index: 146

Text #17

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.2238. Min Act: -0.1700

Data Index: 5768291 (Open Web Text)

Max Activating Token Index: 353

Text #18

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.1905. Min Act: -0.1700

Data Index: 5148109 (Open Web Text)

Max Activating Token Index: 862

Text #19

Max Range: 5.5323. Min Range: -5.5323

Max Act: 5.1982. Min Act: -0.1700

Data Index: 500015 (Open Web Text)

Max Activating Token Index: 189