Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1538 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.9549. Min Act: -0.1700

Data Index: 1485463 (Open Web Text)

Max Activating Token Index: 540

Text #1

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.9000. Min Act: -0.1700

Data Index: 8130423 (Open Web Text)

Max Activating Token Index: 9

Text #2

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.8802. Min Act: -0.1700

Data Index: 3206769 (Open Web Text)

Max Activating Token Index: 82

Text #3

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.8643. Min Act: -0.1700

Data Index: 3338398 (Open Web Text)

Max Activating Token Index: 92

Text #4

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.8507. Min Act: -0.1700

Data Index: 1125189 (Open Web Text)

Max Activating Token Index: 937

Text #5

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.8272. Min Act: -0.1700

Data Index: 3182921 (Open Web Text)

Max Activating Token Index: 160

Text #6

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.8120. Min Act: -0.1700

Data Index: 1246641 (Open Web Text)

Max Activating Token Index: 220

Text #7

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.7915. Min Act: -0.1700

Data Index: 6440225 (Open Web Text)

Max Activating Token Index: 170

Text #8

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.8428. Min Act: -0.1700

Data Index: 1983459 (Open Web Text)

Max Activating Token Index: 35

Text #9

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.7976. Min Act: -0.1700

Data Index: 1421832 (Open Web Text)

Max Activating Token Index: 101

Text #10

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.8096. Min Act: -0.1700

Data Index: 8453398 (Open Web Text)

Max Activating Token Index: 799

Text #11

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.7459. Min Act: -0.1700

Data Index: 6141508 (Open Web Text)

Max Activating Token Index: 860

Text #12

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.7716. Min Act: -0.1700

Data Index: 1748123 (Open Web Text)

Max Activating Token Index: 807

Text #13

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.7554. Min Act: -0.1700

Data Index: 4666467 (Open Web Text)

Max Activating Token Index: 334

Text #14

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.7343. Min Act: -0.1700

Data Index: 3213982 (Open Web Text)

Max Activating Token Index: 710

Text #15

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.7380. Min Act: -0.1700

Data Index: 7359844 (Open Web Text)

Max Activating Token Index: 996

Text #16

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.7626. Min Act: -0.1700

Data Index: 5534016 (Open Web Text)

Max Activating Token Index: 432

Text #17

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.7149. Min Act: -0.1700

Data Index: 5739518 (Open Web Text)

Max Activating Token Index: 728

Text #18

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.6895. Min Act: -0.1700

Data Index: 4003874 (Open Web Text)

Max Activating Token Index: 887

Text #19

Max Range: 3.9549. Min Range: -3.9549

Max Act: 3.7248. Min Act: -0.1700

Data Index: 4348444 (Open Web Text)

Max Activating Token Index: 113