Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1658 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.0692. Min Range: -6.0692

Max Act: 6.0692. Min Act: -0.1700

Data Index: 6931949 (Open Web Text)

Max Activating Token Index: 316

Text #1

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.9875. Min Act: -0.1700

Data Index: 4391244 (Open Web Text)

Max Activating Token Index: 653

Text #2

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.9721. Min Act: -0.1700

Data Index: 5551994 (Open Web Text)

Max Activating Token Index: 873

Text #3

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.9282. Min Act: -0.1700

Data Index: 228265 (Open Web Text)

Max Activating Token Index: 808

Text #4

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.9656. Min Act: -0.1700

Data Index: 7578588 (Open Web Text)

Max Activating Token Index: 811

Text #5

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.8972. Min Act: -0.1700

Data Index: 7437099 (Open Web Text)

Max Activating Token Index: 938

Text #6

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.9090. Min Act: -0.1700

Data Index: 3680147 (Open Web Text)

Max Activating Token Index: 314

Text #7

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.9105. Min Act: -0.1700

Data Index: 8685205 (Open Web Text)

Max Activating Token Index: 448

Text #8

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.9004. Min Act: -0.1700

Data Index: 1569799 (Open Web Text)

Max Activating Token Index: 166

Text #9

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.8519. Min Act: -0.1700

Data Index: 5332032 (Open Web Text)

Max Activating Token Index: 310

Text #10

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.8441. Min Act: -0.1700

Data Index: 4704219 (Open Web Text)

Max Activating Token Index: 811

Text #11

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.8253. Min Act: -0.1700

Data Index: 2397552 (Open Web Text)

Max Activating Token Index: 824

Text #12

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.8460. Min Act: -0.1700

Data Index: 6511778 (Open Web Text)

Max Activating Token Index: 39

Text #13

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.8429. Min Act: -0.1700

Data Index: 8564206 (Open Web Text)

Max Activating Token Index: 164

Text #14

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.8509. Min Act: -0.1700

Data Index: 6946073 (Open Web Text)

Max Activating Token Index: 734

Text #15

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.7825. Min Act: -0.1700

Data Index: 344642 (Open Web Text)

Max Activating Token Index: 798

Text #16

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.7918. Min Act: -0.1700

Data Index: 5451978 (Open Web Text)

Max Activating Token Index: 168

Text #17

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.8039. Min Act: -0.1700

Data Index: 6859104 (Open Web Text)

Max Activating Token Index: 714

Text #18

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.8056. Min Act: -0.1700

Data Index: 1079683 (Open Web Text)

Max Activating Token Index: 858

Text #19

Max Range: 6.0692. Min Range: -6.0692

Max Act: 5.7812. Min Act: -0.1700

Data Index: 6931946 (Open Web Text)

Max Activating Token Index: 336