Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2731 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.3074. Min Act: -0.0835

Data Index: 7521152 (Open Web Text)

Max Activating Token Index: 366

Text #1

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2755. Min Act: -0.0345

Data Index: 2052330 (Open Web Text)

Max Activating Token Index: 760

Text #2

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2612. Min Act: -0.0728

Data Index: 219971 (Open Web Text)

Max Activating Token Index: 732

Text #3

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2581. Min Act: -0.0537

Data Index: 3801818 (Open Web Text)

Max Activating Token Index: 717

Text #4

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2541. Min Act: -0.0932

Data Index: 4861366 (Open Web Text)

Max Activating Token Index: 733

Text #5

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2519. Min Act: -0.1177

Data Index: 7963501 (Open Web Text)

Max Activating Token Index: 187

Text #6

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2474. Min Act: -0.0624

Data Index: 3875324 (Open Web Text)

Max Activating Token Index: 867

Text #7

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2402. Min Act: -0.0436

Data Index: 2208135 (Open Web Text)

Max Activating Token Index: 884

Text #8

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2531. Min Act: -0.1458

Data Index: 6987310 (Open Web Text)

Max Activating Token Index: 527

Text #9

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2416. Min Act: -0.0690

Data Index: 2771107 (Open Web Text)

Max Activating Token Index: 773

Text #10

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2462. Min Act: -0.1020

Data Index: 5824266 (Open Web Text)

Max Activating Token Index: 812

Text #11

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2459. Min Act: -0.0478

Data Index: 3776046 (Open Web Text)

Max Activating Token Index: 185

Text #12

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2467. Min Act: -0.0755

Data Index: 5536770 (Open Web Text)

Max Activating Token Index: 692

Text #13

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2455. Min Act: -0.0445

Data Index: 6122390 (Open Web Text)

Max Activating Token Index: 532

Text #14

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2448. Min Act: -0.0585

Data Index: 8241516 (Open Web Text)

Max Activating Token Index: 565

Text #15

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2412. Min Act: -0.0938

Data Index: 1756358 (Open Web Text)

Max Activating Token Index: 723

Text #16

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2395. Min Act: -0.1303

Data Index: 4293992 (Open Web Text)

Max Activating Token Index: 120

Text #17

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2416. Min Act: -0.0744

Data Index: 2057020 (Open Web Text)

Max Activating Token Index: 799

Text #18

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2317. Min Act: -0.1070

Data Index: 3874912 (Open Web Text)

Max Activating Token Index: 210

Text #19

Max Range: 1.3074. Min Range: -1.3074

Max Act: 1.2349. Min Act: -0.0955

Data Index: 4452199 (Open Web Text)

Max Activating Token Index: 523