Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 114 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.7252. Min Act: -0.1700

Data Index: 6670795 (Open Web Text)

Max Activating Token Index: 931

Text #1

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.7382. Min Act: -0.1700

Data Index: 130069 (Open Web Text)

Max Activating Token Index: 310

Text #2

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.7440. Min Act: -0.1700

Data Index: 255342 (Open Web Text)

Max Activating Token Index: 833

Text #3

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.6546. Min Act: -0.1700

Data Index: 8456154 (Open Web Text)

Max Activating Token Index: 253

Text #4

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.6461. Min Act: -0.1700

Data Index: 4639064 (Open Web Text)

Max Activating Token Index: 879

Text #5

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.6608. Min Act: -0.1700

Data Index: 1424077 (Open Web Text)

Max Activating Token Index: 763

Text #6

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.5933. Min Act: -0.1700

Data Index: 7020144 (Open Web Text)

Max Activating Token Index: 31

Text #7

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.6227. Min Act: -0.1700

Data Index: 7354770 (Open Web Text)

Max Activating Token Index: 312

Text #8

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.5885. Min Act: -0.1700

Data Index: 8140997 (Open Web Text)

Max Activating Token Index: 636

Text #9

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.6010. Min Act: -0.1700

Data Index: 3942046 (Open Web Text)

Max Activating Token Index: 831

Text #10

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.5678. Min Act: -0.1700

Data Index: 1092439 (Open Web Text)

Max Activating Token Index: 54

Text #11

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.5835. Min Act: -0.1700

Data Index: 774061 (Open Web Text)

Max Activating Token Index: 215

Text #12

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.6219. Min Act: -0.1700

Data Index: 1044182 (Open Web Text)

Max Activating Token Index: 875

Text #13

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.5369. Min Act: -0.1700

Data Index: 6092345 (Open Web Text)

Max Activating Token Index: 301

Text #14

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.5636. Min Act: -0.1700

Data Index: 43273 (Open Web Text)

Max Activating Token Index: 349

Text #15

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.5713. Min Act: -0.1700

Data Index: 2649845 (Open Web Text)

Max Activating Token Index: 441

Text #16

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.5282. Min Act: -0.1700

Data Index: 238036 (Open Web Text)

Max Activating Token Index: 897

Text #17

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.5389. Min Act: -0.1700

Data Index: 5536692 (Open Web Text)

Max Activating Token Index: 806

Text #18

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.5344. Min Act: -0.1700

Data Index: 1990716 (Open Web Text)

Max Activating Token Index: 87

Text #19

Max Range: 4.7440. Min Range: -4.7440

Max Act: 4.5160. Min Act: -0.1700

Data Index: 4010238 (Open Web Text)

Max Activating Token Index: 583