Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1510 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7594. Min Act: -0.1700

Data Index: 6548830 (Open Web Text)

Max Activating Token Index: 922

Text #1

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7445. Min Act: -0.1700

Data Index: 6582282 (Open Web Text)

Max Activating Token Index: 992

Text #2

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7598. Min Act: -0.1700

Data Index: 6804488 (Open Web Text)

Max Activating Token Index: 705

Text #3

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7575. Min Act: -0.1700

Data Index: 7673705 (Open Web Text)

Max Activating Token Index: 40

Text #4

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7321. Min Act: -0.1700

Data Index: 6128674 (Open Web Text)

Max Activating Token Index: 1022

Text #5

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7783. Min Act: -0.1700

Data Index: 4597014 (Open Web Text)

Max Activating Token Index: 541

Text #6

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7399. Min Act: -0.1700

Data Index: 8621990 (Open Web Text)

Max Activating Token Index: 545

Text #7

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7080. Min Act: -0.1700

Data Index: 522685 (Open Web Text)

Max Activating Token Index: 696

Text #8

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7149. Min Act: -0.1700

Data Index: 6099181 (Open Web Text)

Max Activating Token Index: 1011

Text #9

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7318. Min Act: -0.1700

Data Index: 8636230 (Open Web Text)

Max Activating Token Index: 987

Text #10

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7258. Min Act: -0.1700

Data Index: 3530592 (Open Web Text)

Max Activating Token Index: 15

Text #11

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7158. Min Act: -0.1700

Data Index: 4627148 (Open Web Text)

Max Activating Token Index: 94

Text #12

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7283. Min Act: -0.1700

Data Index: 4525428 (Open Web Text)

Max Activating Token Index: 20

Text #13

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7489. Min Act: -0.1700

Data Index: 7792219 (Open Web Text)

Max Activating Token Index: 957

Text #14

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7199. Min Act: -0.1700

Data Index: 3935282 (Open Web Text)

Max Activating Token Index: 97

Text #15

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.6961. Min Act: -0.1700

Data Index: 5677797 (Open Web Text)

Max Activating Token Index: 893

Text #16

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7033. Min Act: -0.1700

Data Index: 2960087 (Open Web Text)

Max Activating Token Index: 67

Text #17

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.6978. Min Act: -0.1700

Data Index: 6267670 (Open Web Text)

Max Activating Token Index: 37

Text #18

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7093. Min Act: -0.1700

Data Index: 3378470 (Open Web Text)

Max Activating Token Index: 746

Text #19

Max Range: 3.7783. Min Range: -3.7783

Max Act: 3.7097. Min Act: -0.1700

Data Index: 559060 (Open Web Text)

Max Activating Token Index: 6