Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1556 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.4397. Min Act: -0.1700

Data Index: 2927887 (Open Web Text)

Max Activating Token Index: 30

Text #1

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.2291. Min Act: -0.1700

Data Index: 6798632 (Open Web Text)

Max Activating Token Index: 832

Text #2

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.1959. Min Act: -0.1700

Data Index: 5420932 (Open Web Text)

Max Activating Token Index: 839

Text #3

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.1377. Min Act: -0.1700

Data Index: 4961693 (Open Web Text)

Max Activating Token Index: 137

Text #4

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.1146. Min Act: -0.1700

Data Index: 8534085 (Open Web Text)

Max Activating Token Index: 182

Text #5

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.1072. Min Act: -0.1700

Data Index: 2961342 (Open Web Text)

Max Activating Token Index: 669

Text #6

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.0816. Min Act: -0.1700

Data Index: 6902299 (Open Web Text)

Max Activating Token Index: 185

Text #7

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.0659. Min Act: -0.1700

Data Index: 3263132 (Open Web Text)

Max Activating Token Index: 357

Text #8

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.0588. Min Act: -0.1700

Data Index: 5863739 (Open Web Text)

Max Activating Token Index: 972

Text #9

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.0535. Min Act: -0.1700

Data Index: 6732578 (Open Web Text)

Max Activating Token Index: 74

Text #10

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.0342. Min Act: -0.1700

Data Index: 6708278 (Open Web Text)

Max Activating Token Index: 128

Text #11

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.0331. Min Act: -0.1700

Data Index: 2456466 (Open Web Text)

Max Activating Token Index: 434

Text #12

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.0013. Min Act: -0.1700

Data Index: 4131777 (Open Web Text)

Max Activating Token Index: 81

Text #13

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.0008. Min Act: -0.1700

Data Index: 1536659 (Open Web Text)

Max Activating Token Index: 850

Text #14

Max Range: 3.4397. Min Range: -3.4397

Max Act: 3.0090. Min Act: -0.1700

Data Index: 6732580 (Open Web Text)

Max Activating Token Index: 447

Text #15

Max Range: 3.4397. Min Range: -3.4397

Max Act: 2.9971. Min Act: -0.1700

Data Index: 1488255 (Open Web Text)

Max Activating Token Index: 728

Text #16

Max Range: 3.4397. Min Range: -3.4397

Max Act: 2.9224. Min Act: -0.1700

Data Index: 1375347 (Open Web Text)

Max Activating Token Index: 60

Text #17

Max Range: 3.4397. Min Range: -3.4397

Max Act: 2.9469. Min Act: -0.1700

Data Index: 5421147 (Open Web Text)

Max Activating Token Index: 289

Text #18

Max Range: 3.4397. Min Range: -3.4397

Max Act: 2.9615. Min Act: -0.1700

Data Index: 1762697 (Open Web Text)

Max Activating Token Index: 528

Text #19

Max Range: 3.4397. Min Range: -3.4397

Max Act: 2.9262. Min Act: -0.1700

Data Index: 202420 (Open Web Text)

Max Activating Token Index: 742