Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2169 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1242. Min Range: -4.1242

Max Act: 4.1242. Min Act: -0.1700

Data Index: 7386120 (Open Web Text)

Max Activating Token Index: 994

Text #1

Max Range: 4.1242. Min Range: -4.1242

Max Act: 4.0804. Min Act: -0.1700

Data Index: 2669405 (Open Web Text)

Max Activating Token Index: 688

Text #2

Max Range: 4.1242. Min Range: -4.1242

Max Act: 4.0885. Min Act: -0.1700

Data Index: 186034 (Open Web Text)

Max Activating Token Index: 425

Text #3

Max Range: 4.1242. Min Range: -4.1242

Max Act: 4.0397. Min Act: -0.1700

Data Index: 1175324 (Open Web Text)

Max Activating Token Index: 875

Text #4

Max Range: 4.1242. Min Range: -4.1242

Max Act: 4.0413. Min Act: -0.1700

Data Index: 6082817 (Open Web Text)

Max Activating Token Index: 31

Text #5

Max Range: 4.1242. Min Range: -4.1242

Max Act: 4.0324. Min Act: -0.1700

Data Index: 3017282 (Open Web Text)

Max Activating Token Index: 824

Text #6

Max Range: 4.1242. Min Range: -4.1242

Max Act: 4.0092. Min Act: -0.1700

Data Index: 5246532 (Open Web Text)

Max Activating Token Index: 517

Text #7

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9576. Min Act: -0.1700

Data Index: 3857093 (Open Web Text)

Max Activating Token Index: 882

Text #8

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9731. Min Act: -0.1700

Data Index: 7070384 (Open Web Text)

Max Activating Token Index: 637

Text #9

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9797. Min Act: -0.1700

Data Index: 2274778 (Open Web Text)

Max Activating Token Index: 981

Text #10

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9849. Min Act: -0.1700

Data Index: 6939961 (Open Web Text)

Max Activating Token Index: 900

Text #11

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9838. Min Act: -0.1700

Data Index: 4302285 (Open Web Text)

Max Activating Token Index: 689

Text #12

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9839. Min Act: -0.1700

Data Index: 7176747 (Open Web Text)

Max Activating Token Index: 266

Text #13

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9849. Min Act: -0.1700

Data Index: 6733659 (Open Web Text)

Max Activating Token Index: 486

Text #14

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9721. Min Act: -0.1700

Data Index: 6351746 (Open Web Text)

Max Activating Token Index: 442

Text #15

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9771. Min Act: -0.1700

Data Index: 8102594 (Open Web Text)

Max Activating Token Index: 654

Text #16

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9492. Min Act: -0.1700

Data Index: 986117 (Open Web Text)

Max Activating Token Index: 606

Text #17

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9453. Min Act: -0.1700

Data Index: 4181093 (Open Web Text)

Max Activating Token Index: 401

Text #18

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9551. Min Act: -0.1700

Data Index: 2710463 (Open Web Text)

Max Activating Token Index: 140

Text #19

Max Range: 4.1242. Min Range: -4.1242

Max Act: 3.9643. Min Act: -0.1700

Data Index: 2843395 (Open Web Text)

Max Activating Token Index: 30