Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 420 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.3940. Min Act: -0.1700

Data Index: 784858 (Open Web Text)

Max Activating Token Index: 517

Text #1

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.4106. Min Act: -0.1700

Data Index: 4432011 (Open Web Text)

Max Activating Token Index: 354

Text #2

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.3349. Min Act: -0.1700

Data Index: 7873818 (Open Web Text)

Max Activating Token Index: 449

Text #3

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.2866. Min Act: -0.1700

Data Index: 4668443 (Open Web Text)

Max Activating Token Index: 35

Text #4

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.2441. Min Act: -0.1700

Data Index: 8307881 (Open Web Text)

Max Activating Token Index: 381

Text #5

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.2570. Min Act: -0.1700

Data Index: 5970821 (Open Web Text)

Max Activating Token Index: 824

Text #6

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.2464. Min Act: -0.1700

Data Index: 6346728 (Open Web Text)

Max Activating Token Index: 559

Text #7

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.2035. Min Act: -0.1700

Data Index: 3771190 (Open Web Text)

Max Activating Token Index: 562

Text #8

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.2323. Min Act: -0.1700

Data Index: 2358144 (Open Web Text)

Max Activating Token Index: 835

Text #9

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.1659. Min Act: -0.1700

Data Index: 2049838 (Open Web Text)

Max Activating Token Index: 320

Text #10

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.1711. Min Act: -0.1700

Data Index: 5695264 (Open Web Text)

Max Activating Token Index: 817

Text #11

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.1842. Min Act: -0.1700

Data Index: 6237508 (Open Web Text)

Max Activating Token Index: 833

Text #12

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.1453. Min Act: -0.1700

Data Index: 6748225 (Open Web Text)

Max Activating Token Index: 823

Text #13

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.1523. Min Act: -0.1700

Data Index: 4533316 (Open Web Text)

Max Activating Token Index: 598

Text #14

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.1059. Min Act: -0.1700

Data Index: 4253464 (Open Web Text)

Max Activating Token Index: 563

Text #15

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.1386. Min Act: -0.1700

Data Index: 868502 (Open Web Text)

Max Activating Token Index: 448

Text #16

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.1368. Min Act: -0.1700

Data Index: 1833059 (Open Web Text)

Max Activating Token Index: 354

Text #17

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.1127. Min Act: -0.1700

Data Index: 882940 (Open Web Text)

Max Activating Token Index: 765

Text #18

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.1298. Min Act: -0.1700

Data Index: 7210105 (Open Web Text)

Max Activating Token Index: 175

Text #19

Max Range: 3.4106. Min Range: -3.4106

Max Act: 3.1141. Min Act: -0.1700

Data Index: 302509 (Open Web Text)

Max Activating Token Index: 188