Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1820 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.7738. Min Act: -0.1700

Data Index: 3898466 (Open Web Text)

Max Activating Token Index: 96

Text #1

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.7495. Min Act: -0.1700

Data Index: 2719367 (Open Web Text)

Max Activating Token Index: 662

Text #2

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.7293. Min Act: -0.1700

Data Index: 2385999 (Open Web Text)

Max Activating Token Index: 50

Text #3

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.7438. Min Act: -0.1700

Data Index: 5695408 (Open Web Text)

Max Activating Token Index: 54

Text #4

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6703. Min Act: -0.1700

Data Index: 7126367 (Open Web Text)

Max Activating Token Index: 401

Text #5

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.7166. Min Act: -0.1700

Data Index: 163430 (Open Web Text)

Max Activating Token Index: 809

Text #6

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.7133. Min Act: -0.1700

Data Index: 6030993 (Open Web Text)

Max Activating Token Index: 1001

Text #7

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6984. Min Act: -0.1700

Data Index: 1924059 (Open Web Text)

Max Activating Token Index: 642

Text #8

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6510. Min Act: -0.1700

Data Index: 6215246 (Open Web Text)

Max Activating Token Index: 543

Text #9

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6684. Min Act: -0.1700

Data Index: 1484745 (Open Web Text)

Max Activating Token Index: 431

Text #10

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6568. Min Act: -0.1700

Data Index: 4648228 (Open Web Text)

Max Activating Token Index: 136

Text #11

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6785. Min Act: -0.1700

Data Index: 5999765 (Open Web Text)

Max Activating Token Index: 844

Text #12

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6532. Min Act: -0.1700

Data Index: 4287942 (Open Web Text)

Max Activating Token Index: 876

Text #13

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6238. Min Act: -0.1700

Data Index: 5028412 (Open Web Text)

Max Activating Token Index: 505

Text #14

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6018. Min Act: -0.1700

Data Index: 4417483 (Open Web Text)

Max Activating Token Index: 61

Text #15

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6002. Min Act: -0.1700

Data Index: 1945579 (Open Web Text)

Max Activating Token Index: 27

Text #16

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6423. Min Act: -0.1700

Data Index: 497613 (Open Web Text)

Max Activating Token Index: 603

Text #17

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.5881. Min Act: -0.1700

Data Index: 6891558 (Open Web Text)

Max Activating Token Index: 363

Text #18

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6106. Min Act: -0.1700

Data Index: 5694014 (Open Web Text)

Max Activating Token Index: 289

Text #19

Max Range: 5.7738. Min Range: -5.7738

Max Act: 5.6289. Min Act: -0.1700

Data Index: 5312437 (Open Web Text)

Max Activating Token Index: 603