Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1511 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.9164. Min Act: -0.1700

Data Index: 3756728 (Open Web Text)

Max Activating Token Index: 489

Text #1

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8916. Min Act: -0.1700

Data Index: 7962436 (Open Web Text)

Max Activating Token Index: 1004

Text #2

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.9011. Min Act: -0.1700

Data Index: 4849242 (Open Web Text)

Max Activating Token Index: 954

Text #3

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8977. Min Act: -0.1700

Data Index: 4568311 (Open Web Text)

Max Activating Token Index: 902

Text #4

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8897. Min Act: -0.1700

Data Index: 6707640 (Open Web Text)

Max Activating Token Index: 576

Text #5

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.9091. Min Act: -0.1700

Data Index: 3355276 (Open Web Text)

Max Activating Token Index: 485

Text #6

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8657. Min Act: -0.1700

Data Index: 8228500 (Open Web Text)

Max Activating Token Index: 1017

Text #7

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8879. Min Act: -0.1700

Data Index: 1719134 (Open Web Text)

Max Activating Token Index: 648

Text #8

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8454. Min Act: -0.1700

Data Index: 4667356 (Open Web Text)

Max Activating Token Index: 483

Text #9

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8760. Min Act: -0.1700

Data Index: 445725 (Open Web Text)

Max Activating Token Index: 229

Text #10

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8141. Min Act: -0.1700

Data Index: 7475209 (Open Web Text)

Max Activating Token Index: 491

Text #11

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8404. Min Act: -0.1700

Data Index: 7018657 (Open Web Text)

Max Activating Token Index: 816

Text #12

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8126. Min Act: -0.1700

Data Index: 5212614 (Open Web Text)

Max Activating Token Index: 277

Text #13

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8578. Min Act: -0.1700

Data Index: 1600219 (Open Web Text)

Max Activating Token Index: 628

Text #14

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.7799. Min Act: -0.1700

Data Index: 3331185 (Open Web Text)

Max Activating Token Index: 912

Text #15

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.7665. Min Act: -0.1700

Data Index: 6584652 (Open Web Text)

Max Activating Token Index: 801

Text #16

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8014. Min Act: -0.1700

Data Index: 2490544 (Open Web Text)

Max Activating Token Index: 955

Text #17

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8017. Min Act: -0.1700

Data Index: 4637620 (Open Web Text)

Max Activating Token Index: 373

Text #18

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.7815. Min Act: -0.1700

Data Index: 37129 (Open Web Text)

Max Activating Token Index: 257

Text #19

Max Range: 3.9164. Min Range: -3.9164

Max Act: 3.8003. Min Act: -0.1700

Data Index: 486623 (Open Web Text)

Max Activating Token Index: 187