Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 298 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0399. Min Range: -4.0399

Max Act: 4.0399. Min Act: -0.1700

Data Index: 8621484 (Open Web Text)

Max Activating Token Index: 571

Text #1

Max Range: 4.0399. Min Range: -4.0399

Max Act: 4.0152. Min Act: -0.1700

Data Index: 3511592 (Open Web Text)

Max Activating Token Index: 801

Text #2

Max Range: 4.0399. Min Range: -4.0399

Max Act: 4.0100. Min Act: -0.1700

Data Index: 5824472 (Open Web Text)

Max Activating Token Index: 163

Text #3

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9860. Min Act: -0.1700

Data Index: 2062976 (Open Web Text)

Max Activating Token Index: 919

Text #4

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9640. Min Act: -0.1700

Data Index: 1615786 (Open Web Text)

Max Activating Token Index: 143

Text #5

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9978. Min Act: -0.1700

Data Index: 2561494 (Open Web Text)

Max Activating Token Index: 145

Text #6

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9561. Min Act: -0.1700

Data Index: 2906393 (Open Web Text)

Max Activating Token Index: 869

Text #7

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9571. Min Act: -0.1700

Data Index: 7042930 (Open Web Text)

Max Activating Token Index: 227

Text #8

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9343. Min Act: -0.1700

Data Index: 2540930 (Open Web Text)

Max Activating Token Index: 960

Text #9

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9424. Min Act: -0.1700

Data Index: 3377400 (Open Web Text)

Max Activating Token Index: 196

Text #10

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9289. Min Act: -0.1700

Data Index: 6370747 (Open Web Text)

Max Activating Token Index: 359

Text #11

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9253. Min Act: -0.1700

Data Index: 6294936 (Open Web Text)

Max Activating Token Index: 826

Text #12

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9126. Min Act: -0.1700

Data Index: 3791371 (Open Web Text)

Max Activating Token Index: 210

Text #13

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9397. Min Act: -0.1700

Data Index: 6283537 (Open Web Text)

Max Activating Token Index: 296

Text #14

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9314. Min Act: -0.1700

Data Index: 2765149 (Open Web Text)

Max Activating Token Index: 856

Text #15

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9456. Min Act: -0.1700

Data Index: 1650979 (Open Web Text)

Max Activating Token Index: 65

Text #16

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9084. Min Act: -0.1700

Data Index: 4901977 (Open Web Text)

Max Activating Token Index: 771

Text #17

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.8799. Min Act: -0.1700

Data Index: 5676161 (Open Web Text)

Max Activating Token Index: 580

Text #18

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9213. Min Act: -0.1700

Data Index: 5815897 (Open Web Text)

Max Activating Token Index: 372

Text #19

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.8873. Min Act: -0.1700

Data Index: 2512138 (Open Web Text)

Max Activating Token Index: 606