Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1545 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2521. Min Range: -5.2521

Max Act: 5.2521. Min Act: -0.1007

Data Index: 5524205 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 5.2521. Min Range: -5.2521

Max Act: 5.1796. Min Act: -0.1485

Data Index: 4501934 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 5.2521. Min Range: -5.2521

Max Act: 5.1508. Min Act: -0.0474

Data Index: 2773234 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 5.2521. Min Range: -5.2521

Max Act: 5.0647. Min Act: -0.0434

Data Index: 3912702 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 5.2521. Min Range: -5.2521

Max Act: 5.0673. Min Act: 0.0115

Data Index: 2633880 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 5.2521. Min Range: -5.2521

Max Act: 5.0635. Min Act: 0.0369

Data Index: 4739656 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 5.2521. Min Range: -5.2521

Max Act: 5.0863. Min Act: -0.0421

Data Index: 1178486 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 5.2521. Min Range: -5.2521

Max Act: 5.1269. Min Act: 0.0376

Data Index: 571785 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 5.2521. Min Range: -5.2521

Max Act: 5.0164. Min Act: -0.0217

Data Index: 2153152 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 5.2521. Min Range: -5.2521

Max Act: 5.0934. Min Act: 0.0204

Data Index: 2565332 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 5.2521. Min Range: -5.2521

Max Act: 5.0700. Min Act: 0.0377

Data Index: 7915991 (Open Web Text)

Max Activating Token Index: 1023

Text #11

Max Range: 5.2521. Min Range: -5.2521

Max Act: 5.0006. Min Act: -0.1217

Data Index: 7344049 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 5.2521. Min Range: -5.2521

Max Act: 4.9578. Min Act: -0.0436

Data Index: 6448107 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 5.2521. Min Range: -5.2521

Max Act: 4.9512. Min Act: -0.1197

Data Index: 8349256 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 5.2521. Min Range: -5.2521

Max Act: 4.9361. Min Act: -0.1016

Data Index: 2029688 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 5.2521. Min Range: -5.2521

Max Act: 4.9884. Min Act: -0.0789

Data Index: 6470895 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 5.2521. Min Range: -5.2521

Max Act: 4.9562. Min Act: -0.0011

Data Index: 5256236 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 5.2521. Min Range: -5.2521

Max Act: 4.9232. Min Act: -0.0445

Data Index: 3100636 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 5.2521. Min Range: -5.2521

Max Act: 4.9731. Min Act: 0.0170

Data Index: 7842230 (Open Web Text)

Max Activating Token Index: 1023

Text #19

Max Range: 5.2521. Min Range: -5.2521

Max Act: 4.9641. Min Act: 0.2560

Data Index: 3646565 (Open Web Text)

Max Activating Token Index: 1023