Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 596 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.9000. Min Act: -0.1653

Data Index: 1774036 (Open Web Text)

Max Activating Token Index: 578

Text #1

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.7724. Min Act: -0.1654

Data Index: 1774039 (Open Web Text)

Max Activating Token Index: 745

Text #2

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.7796. Min Act: -0.1699

Data Index: 1774042 (Open Web Text)

Max Activating Token Index: 81

Text #3

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.5735. Min Act: -0.1681

Data Index: 1774026 (Open Web Text)

Max Activating Token Index: 69

Text #4

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.5074. Min Act: -0.1688

Data Index: 1774028 (Open Web Text)

Max Activating Token Index: 79

Text #5

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.5488. Min Act: -0.1695

Data Index: 1774030 (Open Web Text)

Max Activating Token Index: 197

Text #6

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.5338. Min Act: -0.1667

Data Index: 1774035 (Open Web Text)

Max Activating Token Index: 519

Text #7

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.4294. Min Act: -0.1580

Data Index: 1774023 (Open Web Text)

Max Activating Token Index: 220

Text #8

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.4129. Min Act: -0.1576

Data Index: 1774024 (Open Web Text)

Max Activating Token Index: 382

Text #9

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.3258. Min Act: -0.1659

Data Index: 1774031 (Open Web Text)

Max Activating Token Index: 950

Text #10

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.3485. Min Act: -0.1505

Data Index: 1774040 (Open Web Text)

Max Activating Token Index: 31

Text #11

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.3195. Min Act: -0.1697

Data Index: 1774041 (Open Web Text)

Max Activating Token Index: 132

Text #12

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.2395. Min Act: -0.1700

Data Index: 1774037 (Open Web Text)

Max Activating Token Index: 458

Text #13

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.2498. Min Act: -0.1376

Data Index: 1774033 (Open Web Text)

Max Activating Token Index: 849

Text #14

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.2560. Min Act: -0.1700

Data Index: 5308472 (Open Web Text)

Max Activating Token Index: 9

Text #15

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.2171. Min Act: -0.1699

Data Index: 1774038 (Open Web Text)

Max Activating Token Index: 645

Text #16

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.1813. Min Act: -0.1594

Data Index: 1774027 (Open Web Text)

Max Activating Token Index: 145

Text #17

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.1972. Min Act: -0.1693

Data Index: 1774025 (Open Web Text)

Max Activating Token Index: 139

Text #18

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.1972. Min Act: -0.1610

Data Index: 1774022 (Open Web Text)

Max Activating Token Index: 74

Text #19

Max Range: 4.9000. Min Range: -4.9000

Max Act: 4.1920. Min Act: -0.1162

Data Index: 1774032 (Open Web Text)

Max Activating Token Index: 975