Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1976 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.9704. Min Act: -0.1700

Data Index: 4549716 (Open Web Text)

Max Activating Token Index: 278

Text #1

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.8344. Min Act: -0.1700

Data Index: 4940979 (Open Web Text)

Max Activating Token Index: 693

Text #2

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.8209. Min Act: -0.1700

Data Index: 968750 (Open Web Text)

Max Activating Token Index: 620

Text #3

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.8138. Min Act: -0.1700

Data Index: 8140778 (Open Web Text)

Max Activating Token Index: 973

Text #4

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7813. Min Act: -0.1700

Data Index: 4549715 (Open Web Text)

Max Activating Token Index: 937

Text #5

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7763. Min Act: -0.1700

Data Index: 7482234 (Open Web Text)

Max Activating Token Index: 174

Text #6

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7466. Min Act: -0.1700

Data Index: 943192 (Open Web Text)

Max Activating Token Index: 665

Text #7

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7603. Min Act: -0.1700

Data Index: 2880546 (Open Web Text)

Max Activating Token Index: 190

Text #8

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7512. Min Act: -0.1700

Data Index: 806431 (Open Web Text)

Max Activating Token Index: 796

Text #9

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7539. Min Act: -0.1700

Data Index: 5888375 (Open Web Text)

Max Activating Token Index: 280

Text #10

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7423. Min Act: -0.1700

Data Index: 7107163 (Open Web Text)

Max Activating Token Index: 7

Text #11

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7338. Min Act: -0.1700

Data Index: 4549717 (Open Web Text)

Max Activating Token Index: 66

Text #12

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7222. Min Act: -0.1700

Data Index: 3571822 (Open Web Text)

Max Activating Token Index: 802

Text #13

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7258. Min Act: -0.1700

Data Index: 86102 (Open Web Text)

Max Activating Token Index: 161

Text #14

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7046. Min Act: -0.1700

Data Index: 8542381 (Open Web Text)

Max Activating Token Index: 477

Text #15

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7097. Min Act: -0.1700

Data Index: 5113450 (Open Web Text)

Max Activating Token Index: 325

Text #16

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7056. Min Act: -0.1700

Data Index: 5861591 (Open Web Text)

Max Activating Token Index: 146

Text #17

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.7109. Min Act: -0.1700

Data Index: 4428785 (Open Web Text)

Max Activating Token Index: 649

Text #18

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.6993. Min Act: -0.1700

Data Index: 7217177 (Open Web Text)

Max Activating Token Index: 536

Text #19

Max Range: 1.9704. Min Range: -1.9704

Max Act: 1.6916. Min Act: -0.1700

Data Index: 2988090 (Open Web Text)

Max Activating Token Index: 766