Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1085 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0355. Min Range: -4.0355

Max Act: 4.0355. Min Act: -0.1700

Data Index: 7020186 (Open Web Text)

Max Activating Token Index: 781

Text #1

Max Range: 4.0355. Min Range: -4.0355

Max Act: 4.0019. Min Act: -0.1700

Data Index: 8244195 (Open Web Text)

Max Activating Token Index: 179

Text #2

Max Range: 4.0355. Min Range: -4.0355

Max Act: 4.0328. Min Act: -0.1700

Data Index: 2956278 (Open Web Text)

Max Activating Token Index: 1009

Text #3

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.9947. Min Act: -0.1700

Data Index: 8688545 (Open Web Text)

Max Activating Token Index: 349

Text #4

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.9673. Min Act: -0.1700

Data Index: 5893913 (Open Web Text)

Max Activating Token Index: 71

Text #5

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.9434. Min Act: -0.1700

Data Index: 5932245 (Open Web Text)

Max Activating Token Index: 808

Text #6

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.9299. Min Act: -0.1700

Data Index: 5157715 (Open Web Text)

Max Activating Token Index: 355

Text #7

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.9358. Min Act: -0.1700

Data Index: 1518749 (Open Web Text)

Max Activating Token Index: 324

Text #8

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.9137. Min Act: -0.1700

Data Index: 189720 (Open Web Text)

Max Activating Token Index: 320

Text #9

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.9152. Min Act: -0.1700

Data Index: 4919834 (Open Web Text)

Max Activating Token Index: 654

Text #10

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.9180. Min Act: -0.1700

Data Index: 8593886 (Open Web Text)

Max Activating Token Index: 749

Text #11

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.9038. Min Act: -0.1700

Data Index: 1455680 (Open Web Text)

Max Activating Token Index: 295

Text #12

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.9018. Min Act: -0.1700

Data Index: 7076745 (Open Web Text)

Max Activating Token Index: 211

Text #13

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.8465. Min Act: -0.1700

Data Index: 6414064 (Open Web Text)

Max Activating Token Index: 64

Text #14

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.8663. Min Act: -0.1700

Data Index: 6097612 (Open Web Text)

Max Activating Token Index: 828

Text #15

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.8626. Min Act: -0.1700

Data Index: 997416 (Open Web Text)

Max Activating Token Index: 386

Text #16

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.8554. Min Act: -0.1700

Data Index: 3017154 (Open Web Text)

Max Activating Token Index: 778

Text #17

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.8307. Min Act: -0.1700

Data Index: 683190 (Open Web Text)

Max Activating Token Index: 33

Text #18

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.8603. Min Act: -0.1700

Data Index: 7073811 (Open Web Text)

Max Activating Token Index: 544

Text #19

Max Range: 4.0355. Min Range: -4.0355

Max Act: 3.8612. Min Act: -0.1700

Data Index: 815727 (Open Web Text)

Max Activating Token Index: 332