Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 373 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.6709. Min Act: -0.1700

Data Index: 327798 (Open Web Text)

Max Activating Token Index: 7

Text #1

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.6897. Min Act: -0.1700

Data Index: 4917340 (Open Web Text)

Max Activating Token Index: 7

Text #2

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.6333. Min Act: -0.1700

Data Index: 5409891 (Open Web Text)

Max Activating Token Index: 16

Text #3

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.6318. Min Act: -0.1700

Data Index: 6661152 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.6219. Min Act: -0.1700

Data Index: 329942 (Open Web Text)

Max Activating Token Index: 5

Text #5

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5660. Min Act: -0.1700

Data Index: 6148597 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5868. Min Act: -0.1700

Data Index: 4359752 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.6019. Min Act: -0.1700

Data Index: 4930061 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5406. Min Act: -0.1700

Data Index: 6761817 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5511. Min Act: -0.1700

Data Index: 5460022 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5497. Min Act: -0.1700

Data Index: 2885858 (Open Web Text)

Max Activating Token Index: 13

Text #11

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5406. Min Act: -0.1700

Data Index: 44017 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5606. Min Act: -0.1700

Data Index: 626828 (Open Web Text)

Max Activating Token Index: 12

Text #13

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5561. Min Act: -0.1700

Data Index: 7481285 (Open Web Text)

Max Activating Token Index: 113

Text #14

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5629. Min Act: -0.1700

Data Index: 1362733 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5437. Min Act: -0.1700

Data Index: 4327889 (Open Web Text)

Max Activating Token Index: 5

Text #16

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5419. Min Act: -0.1700

Data Index: 4285624 (Open Web Text)

Max Activating Token Index: 37

Text #17

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5406. Min Act: -0.1700

Data Index: 4019625 (Open Web Text)

Max Activating Token Index: 3

Text #18

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5505. Min Act: -0.1700

Data Index: 3931149 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 3.6897. Min Range: -3.6897

Max Act: 3.5466. Min Act: -0.1700

Data Index: 6205994 (Open Web Text)

Max Activating Token Index: 40