Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 72 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.9848. Min Act: -0.1700

Data Index: 1656289 (Open Web Text)

Max Activating Token Index: 811

Text #1

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.9190. Min Act: -0.1700

Data Index: 8242556 (Open Web Text)

Max Activating Token Index: 883

Text #2

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.8666. Min Act: -0.1700

Data Index: 8166909 (Open Web Text)

Max Activating Token Index: 609

Text #3

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.8903. Min Act: -0.1700

Data Index: 4906982 (Open Web Text)

Max Activating Token Index: 765

Text #4

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.8652. Min Act: -0.1700

Data Index: 8039013 (Open Web Text)

Max Activating Token Index: 685

Text #5

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.8178. Min Act: -0.1700

Data Index: 2568651 (Open Web Text)

Max Activating Token Index: 415

Text #6

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.7942. Min Act: -0.1700

Data Index: 6348309 (Open Web Text)

Max Activating Token Index: 742

Text #7

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.7918. Min Act: -0.1700

Data Index: 3125303 (Open Web Text)

Max Activating Token Index: 870

Text #8

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.8371. Min Act: -0.1700

Data Index: 734862 (Open Web Text)

Max Activating Token Index: 941

Text #9

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.8232. Min Act: -0.1700

Data Index: 7882975 (Open Web Text)

Max Activating Token Index: 331

Text #10

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.8250. Min Act: -0.1700

Data Index: 981329 (Open Web Text)

Max Activating Token Index: 59

Text #11

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.7728. Min Act: -0.1700

Data Index: 1885527 (Open Web Text)

Max Activating Token Index: 1005

Text #12

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.7626. Min Act: -0.1700

Data Index: 3135637 (Open Web Text)

Max Activating Token Index: 867

Text #13

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.8264. Min Act: -0.1700

Data Index: 8799184 (Open Web Text)

Max Activating Token Index: 493

Text #14

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.7613. Min Act: -0.1700

Data Index: 270127 (Open Web Text)

Max Activating Token Index: 431

Text #15

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.7919. Min Act: -0.1700

Data Index: 6494266 (Open Web Text)

Max Activating Token Index: 22

Text #16

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.7957. Min Act: -0.1700

Data Index: 5886213 (Open Web Text)

Max Activating Token Index: 142

Text #17

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.7855. Min Act: -0.1700

Data Index: 8548255 (Open Web Text)

Max Activating Token Index: 556

Text #18

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.7407. Min Act: -0.1700

Data Index: 273512 (Open Web Text)

Max Activating Token Index: 609

Text #19

Max Range: 3.9848. Min Range: -3.9848

Max Act: 3.7257. Min Act: -0.1700

Data Index: 5150388 (Open Web Text)

Max Activating Token Index: 84