Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2540 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0504. Min Range: -4.0504

Max Act: 4.0504. Min Act: -0.1700

Data Index: 3858855 (Open Web Text)

Max Activating Token Index: 239

Text #1

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.9720. Min Act: -0.1700

Data Index: 7423330 (Open Web Text)

Max Activating Token Index: 323

Text #2

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.8994. Min Act: -0.1700

Data Index: 8265891 (Open Web Text)

Max Activating Token Index: 151

Text #3

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.9090. Min Act: -0.1700

Data Index: 7423328 (Open Web Text)

Max Activating Token Index: 199

Text #4

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.8956. Min Act: -0.1700

Data Index: 7423323 (Open Web Text)

Max Activating Token Index: 339

Text #5

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.8999. Min Act: -0.1700

Data Index: 7649867 (Open Web Text)

Max Activating Token Index: 66

Text #6

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.8859. Min Act: -0.1700

Data Index: 7423322 (Open Web Text)

Max Activating Token Index: 482

Text #7

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.8686. Min Act: -0.1700

Data Index: 7423347 (Open Web Text)

Max Activating Token Index: 74

Text #8

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.8556. Min Act: -0.1700

Data Index: 5304530 (Open Web Text)

Max Activating Token Index: 70

Text #9

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.8835. Min Act: -0.1700

Data Index: 4967273 (Open Web Text)

Max Activating Token Index: 230

Text #10

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.8318. Min Act: -0.1700

Data Index: 2284759 (Open Web Text)

Max Activating Token Index: 360

Text #11

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.8427. Min Act: -0.1700

Data Index: 2284753 (Open Web Text)

Max Activating Token Index: 29

Text #12

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.8007. Min Act: -0.1700

Data Index: 2284757 (Open Web Text)

Max Activating Token Index: 62

Text #13

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.8237. Min Act: -0.1700

Data Index: 7824941 (Open Web Text)

Max Activating Token Index: 336

Text #14

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.8049. Min Act: -0.1700

Data Index: 2480155 (Open Web Text)

Max Activating Token Index: 660

Text #15

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.7768. Min Act: -0.1700

Data Index: 4585585 (Open Web Text)

Max Activating Token Index: 171

Text #16

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.7719. Min Act: -0.1700

Data Index: 5285634 (Open Web Text)

Max Activating Token Index: 102

Text #17

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.7873. Min Act: -0.1700

Data Index: 5285622 (Open Web Text)

Max Activating Token Index: 383

Text #18

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.7835. Min Act: -0.1700

Data Index: 2284761 (Open Web Text)

Max Activating Token Index: 146

Text #19

Max Range: 4.0504. Min Range: -4.0504

Max Act: 3.7304. Min Act: -0.1700

Data Index: 2840549 (Open Web Text)

Max Activating Token Index: 77