Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1022 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.3795. Min Act: -0.1700

Data Index: 3198540 (Open Web Text)

Max Activating Token Index: 527

Text #1

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.3187. Min Act: -0.1700

Data Index: 3045870 (Open Web Text)

Max Activating Token Index: 654

Text #2

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.3242. Min Act: -0.1700

Data Index: 1813081 (Open Web Text)

Max Activating Token Index: 699

Text #3

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2494. Min Act: -0.1700

Data Index: 3069794 (Open Web Text)

Max Activating Token Index: 471

Text #4

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2435. Min Act: -0.1700

Data Index: 3129513 (Open Web Text)

Max Activating Token Index: 731

Text #5

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2731. Min Act: -0.1700

Data Index: 3526805 (Open Web Text)

Max Activating Token Index: 371

Text #6

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2420. Min Act: -0.1700

Data Index: 6056677 (Open Web Text)

Max Activating Token Index: 677

Text #7

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2691. Min Act: -0.1700

Data Index: 7902608 (Open Web Text)

Max Activating Token Index: 198

Text #8

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2141. Min Act: -0.1700

Data Index: 2165822 (Open Web Text)

Max Activating Token Index: 771

Text #9

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2133. Min Act: -0.1700

Data Index: 2242027 (Open Web Text)

Max Activating Token Index: 573

Text #10

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2228. Min Act: -0.1700

Data Index: 8185425 (Open Web Text)

Max Activating Token Index: 121

Text #11

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2284. Min Act: -0.1700

Data Index: 868896 (Open Web Text)

Max Activating Token Index: 619

Text #12

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2208. Min Act: -0.1700

Data Index: 7789680 (Open Web Text)

Max Activating Token Index: 789

Text #13

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2134. Min Act: -0.1700

Data Index: 7491632 (Open Web Text)

Max Activating Token Index: 703

Text #14

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2293. Min Act: -0.1700

Data Index: 7358504 (Open Web Text)

Max Activating Token Index: 556

Text #15

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2206. Min Act: -0.1700

Data Index: 6411164 (Open Web Text)

Max Activating Token Index: 194

Text #16

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2328. Min Act: -0.1700

Data Index: 765445 (Open Web Text)

Max Activating Token Index: 457

Text #17

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2274. Min Act: -0.1700

Data Index: 5792993 (Open Web Text)

Max Activating Token Index: 242

Text #18

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2524. Min Act: -0.1700

Data Index: 4759217 (Open Web Text)

Max Activating Token Index: 623

Text #19

Max Range: 4.3795. Min Range: -4.3795

Max Act: 4.2181. Min Act: -0.1700

Data Index: 1029736 (Open Web Text)

Max Activating Token Index: 622