Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 12 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0834. Min Range: -4.0834

Max Act: 4.0834. Min Act: -0.1700

Data Index: 8678801 (Open Web Text)

Max Activating Token Index: 715

Text #1

Max Range: 4.0834. Min Range: -4.0834

Max Act: 4.0368. Min Act: -0.1700

Data Index: 7926037 (Open Web Text)

Max Activating Token Index: 682

Text #2

Max Range: 4.0834. Min Range: -4.0834

Max Act: 4.0048. Min Act: -0.1700

Data Index: 1542819 (Open Web Text)

Max Activating Token Index: 641

Text #3

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.8517. Min Act: -0.1700

Data Index: 1029147 (Open Web Text)

Max Activating Token Index: 825

Text #4

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.8474. Min Act: -0.1700

Data Index: 8371457 (Open Web Text)

Max Activating Token Index: 880

Text #5

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.8228. Min Act: -0.1700

Data Index: 8558888 (Open Web Text)

Max Activating Token Index: 716

Text #6

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.7626. Min Act: -0.1700

Data Index: 4319212 (Open Web Text)

Max Activating Token Index: 580

Text #7

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.8044. Min Act: -0.1700

Data Index: 7673995 (Open Web Text)

Max Activating Token Index: 725

Text #8

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.7802. Min Act: -0.1700

Data Index: 6919339 (Open Web Text)

Max Activating Token Index: 707

Text #9

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.8097. Min Act: -0.1700

Data Index: 3012598 (Open Web Text)

Max Activating Token Index: 620

Text #10

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.7838. Min Act: -0.1700

Data Index: 5908615 (Open Web Text)

Max Activating Token Index: 263

Text #11

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.7999. Min Act: -0.1700

Data Index: 6053264 (Open Web Text)

Max Activating Token Index: 590

Text #12

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.7351. Min Act: -0.1700

Data Index: 3163963 (Open Web Text)

Max Activating Token Index: 568

Text #13

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.7434. Min Act: -0.1700

Data Index: 3673259 (Open Web Text)

Max Activating Token Index: 564

Text #14

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.7381. Min Act: -0.1700

Data Index: 8464759 (Open Web Text)

Max Activating Token Index: 618

Text #15

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.7451. Min Act: -0.1700

Data Index: 7909984 (Open Web Text)

Max Activating Token Index: 890

Text #16

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.7520. Min Act: -0.1700

Data Index: 4859492 (Open Web Text)

Max Activating Token Index: 681

Text #17

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.7461. Min Act: -0.1700

Data Index: 7454054 (Open Web Text)

Max Activating Token Index: 793

Text #18

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.7360. Min Act: -0.1700

Data Index: 8727244 (Open Web Text)

Max Activating Token Index: 659

Text #19

Max Range: 4.0834. Min Range: -4.0834

Max Act: 3.7287. Min Act: -0.1700

Data Index: 6619893 (Open Web Text)

Max Activating Token Index: 965