Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2903 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.4228. Min Act: -0.1700

Data Index: 4881579 (Open Web Text)

Max Activating Token Index: 630

Text #1

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3908. Min Act: -0.1700

Data Index: 3116668 (Open Web Text)

Max Activating Token Index: 424

Text #2

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3819. Min Act: -0.1700

Data Index: 8192176 (Open Web Text)

Max Activating Token Index: 145

Text #3

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3752. Min Act: -0.1700

Data Index: 4918015 (Open Web Text)

Max Activating Token Index: 664

Text #4

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3761. Min Act: -0.1700

Data Index: 4918017 (Open Web Text)

Max Activating Token Index: 712

Text #5

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3669. Min Act: -0.1700

Data Index: 7309224 (Open Web Text)

Max Activating Token Index: 996

Text #6

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3629. Min Act: -0.1700

Data Index: 7831651 (Open Web Text)

Max Activating Token Index: 243

Text #7

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3622. Min Act: -0.1700

Data Index: 5021800 (Open Web Text)

Max Activating Token Index: 405

Text #8

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3599. Min Act: -0.1700

Data Index: 2133883 (Open Web Text)

Max Activating Token Index: 123

Text #9

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3552. Min Act: -0.1700

Data Index: 4308416 (Open Web Text)

Max Activating Token Index: 253

Text #10

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3559. Min Act: -0.1700

Data Index: 2800119 (Open Web Text)

Max Activating Token Index: 374

Text #11

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3584. Min Act: -0.1700

Data Index: 3202102 (Open Web Text)

Max Activating Token Index: 567

Text #12

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3485. Min Act: -0.1700

Data Index: 6731073 (Open Web Text)

Max Activating Token Index: 459

Text #13

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3556. Min Act: -0.1700

Data Index: 5343625 (Open Web Text)

Max Activating Token Index: 750

Text #14

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3419. Min Act: -0.1700

Data Index: 5021801 (Open Web Text)

Max Activating Token Index: 1022

Text #15

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3395. Min Act: -0.1700

Data Index: 4881580 (Open Web Text)

Max Activating Token Index: 159

Text #16

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3422. Min Act: -0.1700

Data Index: 6493622 (Open Web Text)

Max Activating Token Index: 533

Text #17

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3468. Min Act: -0.1700

Data Index: 3175240 (Open Web Text)

Max Activating Token Index: 541

Text #18

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3317. Min Act: -0.1696

Data Index: 588573 (Open Web Text)

Max Activating Token Index: 703

Text #19

Max Range: 1.4228. Min Range: -1.4228

Max Act: 1.3390. Min Act: -0.1700

Data Index: 4865698 (Open Web Text)

Max Activating Token Index: 527