Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 65 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.6008. Min Act: -0.1700

Data Index: 7615936 (Open Web Text)

Max Activating Token Index: 175

Text #1

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.5028. Min Act: -0.1700

Data Index: 2356263 (Open Web Text)

Max Activating Token Index: 898

Text #2

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.4846. Min Act: -0.1700

Data Index: 2741882 (Open Web Text)

Max Activating Token Index: 7

Text #3

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.4643. Min Act: -0.1700

Data Index: 2735970 (Open Web Text)

Max Activating Token Index: 41

Text #4

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.4535. Min Act: -0.1700

Data Index: 6628811 (Open Web Text)

Max Activating Token Index: 343

Text #5

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3692. Min Act: -0.1700

Data Index: 4455926 (Open Web Text)

Max Activating Token Index: 87

Text #6

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3437. Min Act: -0.1700

Data Index: 2180862 (Open Web Text)

Max Activating Token Index: 519

Text #7

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3290. Min Act: -0.1700

Data Index: 6193498 (Open Web Text)

Max Activating Token Index: 941

Text #8

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3553. Min Act: -0.1700

Data Index: 1685333 (Open Web Text)

Max Activating Token Index: 9

Text #9

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3485. Min Act: -0.1700

Data Index: 8581846 (Open Web Text)

Max Activating Token Index: 542

Text #10

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3387. Min Act: -0.1700

Data Index: 408556 (Open Web Text)

Max Activating Token Index: 7

Text #11

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3612. Min Act: -0.1700

Data Index: 5543240 (Open Web Text)

Max Activating Token Index: 41

Text #12

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3271. Min Act: -0.1700

Data Index: 7291475 (Open Web Text)

Max Activating Token Index: 947

Text #13

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3362. Min Act: -0.1700

Data Index: 715056 (Open Web Text)

Max Activating Token Index: 239

Text #14

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3463. Min Act: -0.1700

Data Index: 6237115 (Open Web Text)

Max Activating Token Index: 973

Text #15

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3145. Min Act: -0.1700

Data Index: 655545 (Open Web Text)

Max Activating Token Index: 7

Text #16

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3466. Min Act: -0.1700

Data Index: 4565095 (Open Web Text)

Max Activating Token Index: 33

Text #17

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3525. Min Act: -0.1700

Data Index: 1100521 (Open Web Text)

Max Activating Token Index: 386

Text #18

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3064. Min Act: -0.1700

Data Index: 1486015 (Open Web Text)

Max Activating Token Index: 96

Text #19

Max Range: 4.6008. Min Range: -4.6008

Max Act: 4.3100. Min Act: -0.1700

Data Index: 3678643 (Open Web Text)

Max Activating Token Index: 50