Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1605 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1883. Min Range: -5.1883

Max Act: 5.1883. Min Act: -0.1700

Data Index: 6808481 (Open Web Text)

Max Activating Token Index: 457

Text #1

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.7577. Min Act: -0.1700

Data Index: 2804479 (Open Web Text)

Max Activating Token Index: 800

Text #2

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.5876. Min Act: -0.1700

Data Index: 4139952 (Open Web Text)

Max Activating Token Index: 396

Text #3

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.5848. Min Act: -0.1700

Data Index: 869543 (Open Web Text)

Max Activating Token Index: 225

Text #4

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.5884. Min Act: -0.1700

Data Index: 6240367 (Open Web Text)

Max Activating Token Index: 980

Text #5

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.5419. Min Act: -0.1700

Data Index: 8308567 (Open Web Text)

Max Activating Token Index: 875

Text #6

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.5485. Min Act: -0.1700

Data Index: 6398435 (Open Web Text)

Max Activating Token Index: 250

Text #7

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.4601. Min Act: -0.1700

Data Index: 6605114 (Open Web Text)

Max Activating Token Index: 484

Text #8

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.4943. Min Act: -0.1700

Data Index: 5311863 (Open Web Text)

Max Activating Token Index: 690

Text #9

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.5062. Min Act: -0.1700

Data Index: 2769867 (Open Web Text)

Max Activating Token Index: 727

Text #10

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.4971. Min Act: -0.1700

Data Index: 3692752 (Open Web Text)

Max Activating Token Index: 497

Text #11

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.5171. Min Act: -0.1700

Data Index: 3219016 (Open Web Text)

Max Activating Token Index: 886

Text #12

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.4467. Min Act: -0.1700

Data Index: 4069060 (Open Web Text)

Max Activating Token Index: 290

Text #13

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.4152. Min Act: -0.1700

Data Index: 4169493 (Open Web Text)

Max Activating Token Index: 465

Text #14

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.4227. Min Act: -0.1700

Data Index: 7840238 (Open Web Text)

Max Activating Token Index: 298

Text #15

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.4413. Min Act: -0.1700

Data Index: 7124909 (Open Web Text)

Max Activating Token Index: 816

Text #16

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.3959. Min Act: -0.1700

Data Index: 821685 (Open Web Text)

Max Activating Token Index: 249

Text #17

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.3913. Min Act: -0.1700

Data Index: 2360221 (Open Web Text)

Max Activating Token Index: 868

Text #18

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.4271. Min Act: -0.1700

Data Index: 4721125 (Open Web Text)

Max Activating Token Index: 541

Text #19

Max Range: 5.1883. Min Range: -5.1883

Max Act: 4.3707. Min Act: -0.1700

Data Index: 3361415 (Open Web Text)

Max Activating Token Index: 757