Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2041 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.9393. Min Act: -0.1700

Data Index: 7016322 (Open Web Text)

Max Activating Token Index: 67

Text #1

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.9107. Min Act: -0.1700

Data Index: 6662321 (Open Web Text)

Max Activating Token Index: 868

Text #2

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.8260. Min Act: -0.1700

Data Index: 5437252 (Open Web Text)

Max Activating Token Index: 1003

Text #3

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.8390. Min Act: -0.1700

Data Index: 7415188 (Open Web Text)

Max Activating Token Index: 23

Text #4

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.7936. Min Act: -0.1700

Data Index: 143881 (Open Web Text)

Max Activating Token Index: 919

Text #5

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.7660. Min Act: -0.1700

Data Index: 3293652 (Open Web Text)

Max Activating Token Index: 59

Text #6

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.7628. Min Act: -0.1700

Data Index: 2487241 (Open Web Text)

Max Activating Token Index: 39

Text #7

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.7544. Min Act: -0.1700

Data Index: 6431507 (Open Web Text)

Max Activating Token Index: 342

Text #8

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.7659. Min Act: -0.1700

Data Index: 5126317 (Open Web Text)

Max Activating Token Index: 17

Text #9

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.7384. Min Act: -0.1700

Data Index: 5162764 (Open Web Text)

Max Activating Token Index: 323

Text #10

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.7088. Min Act: -0.1700

Data Index: 8426393 (Open Web Text)

Max Activating Token Index: 68

Text #11

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.7309. Min Act: -0.1700

Data Index: 2995079 (Open Web Text)

Max Activating Token Index: 541

Text #12

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.7125. Min Act: -0.1700

Data Index: 2039079 (Open Web Text)

Max Activating Token Index: 28

Text #13

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.7388. Min Act: -0.1700

Data Index: 3710125 (Open Web Text)

Max Activating Token Index: 856

Text #14

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.6924. Min Act: -0.1700

Data Index: 7246601 (Open Web Text)

Max Activating Token Index: 536

Text #15

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.6801. Min Act: -0.1700

Data Index: 2066359 (Open Web Text)

Max Activating Token Index: 64

Text #16

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.6621. Min Act: -0.1700

Data Index: 1234281 (Open Web Text)

Max Activating Token Index: 385

Text #17

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.7082. Min Act: -0.1700

Data Index: 7499470 (Open Web Text)

Max Activating Token Index: 22

Text #18

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.7145. Min Act: -0.1700

Data Index: 160376 (Open Web Text)

Max Activating Token Index: 113

Text #19

Max Range: 3.9393. Min Range: -3.9393

Max Act: 3.6622. Min Act: -0.1700

Data Index: 1055302 (Open Web Text)

Max Activating Token Index: 359