Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1228 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.9059. Min Act: -0.1700

Data Index: 2537267 (Open Web Text)

Max Activating Token Index: 532

Text #1

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.8329. Min Act: -0.1700

Data Index: 7847421 (Open Web Text)

Max Activating Token Index: 455

Text #2

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.7729. Min Act: -0.1700

Data Index: 3614114 (Open Web Text)

Max Activating Token Index: 36

Text #3

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.7586. Min Act: -0.1700

Data Index: 2537276 (Open Web Text)

Max Activating Token Index: 493

Text #4

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.7439. Min Act: -0.1700

Data Index: 8129279 (Open Web Text)

Max Activating Token Index: 27

Text #5

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.7206. Min Act: -0.1700

Data Index: 6393974 (Open Web Text)

Max Activating Token Index: 132

Text #6

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.6980. Min Act: -0.1700

Data Index: 3337951 (Open Web Text)

Max Activating Token Index: 27

Text #7

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.6511. Min Act: -0.1700

Data Index: 6603085 (Open Web Text)

Max Activating Token Index: 833

Text #8

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.6552. Min Act: -0.1700

Data Index: 6242905 (Open Web Text)

Max Activating Token Index: 555

Text #9

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.6483. Min Act: -0.1700

Data Index: 3099942 (Open Web Text)

Max Activating Token Index: 542

Text #10

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.6312. Min Act: -0.1700

Data Index: 1498772 (Open Web Text)

Max Activating Token Index: 604

Text #11

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.6347. Min Act: -0.1700

Data Index: 938437 (Open Web Text)

Max Activating Token Index: 649

Text #12

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.6350. Min Act: -0.1700

Data Index: 3155035 (Open Web Text)

Max Activating Token Index: 182

Text #13

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.6040. Min Act: -0.1700

Data Index: 3550101 (Open Web Text)

Max Activating Token Index: 417

Text #14

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.5909. Min Act: -0.1700

Data Index: 4433890 (Open Web Text)

Max Activating Token Index: 604

Text #15

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.6002. Min Act: -0.1700

Data Index: 5484551 (Open Web Text)

Max Activating Token Index: 338

Text #16

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.6031. Min Act: -0.1700

Data Index: 6580569 (Open Web Text)

Max Activating Token Index: 284

Text #17

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.5795. Min Act: -0.1700

Data Index: 8186354 (Open Web Text)

Max Activating Token Index: 23

Text #18

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.5861. Min Act: -0.1700

Data Index: 4374397 (Open Web Text)

Max Activating Token Index: 456

Text #19

Max Range: 3.9059. Min Range: -3.9059

Max Act: 3.5808. Min Act: -0.1700

Data Index: 2757560 (Open Web Text)

Max Activating Token Index: 69