Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2569 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4928. Min Act: -0.1700

Data Index: 3299776 (Open Web Text)

Max Activating Token Index: 13

Text #1

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4916. Min Act: -0.1700

Data Index: 3369398 (Open Web Text)

Max Activating Token Index: 276

Text #2

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4603. Min Act: -0.1700

Data Index: 2385266 (Open Web Text)

Max Activating Token Index: 757

Text #3

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4186. Min Act: -0.1700

Data Index: 6741636 (Open Web Text)

Max Activating Token Index: 10

Text #4

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4423. Min Act: -0.1700

Data Index: 2575043 (Open Web Text)

Max Activating Token Index: 115

Text #5

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4485. Min Act: -0.1700

Data Index: 2458306 (Open Web Text)

Max Activating Token Index: 347

Text #6

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4203. Min Act: -0.1700

Data Index: 329091 (Open Web Text)

Max Activating Token Index: 552

Text #7

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4312. Min Act: -0.1700

Data Index: 559894 (Open Web Text)

Max Activating Token Index: 7

Text #8

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4476. Min Act: -0.1700

Data Index: 6802350 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4473. Min Act: -0.1700

Data Index: 2748810 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4382. Min Act: -0.1700

Data Index: 5261160 (Open Web Text)

Max Activating Token Index: 315

Text #11

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4483. Min Act: -0.1700

Data Index: 4684540 (Open Web Text)

Max Activating Token Index: 37

Text #12

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4350. Min Act: -0.1700

Data Index: 1456196 (Open Web Text)

Max Activating Token Index: 8

Text #13

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4282. Min Act: -0.1700

Data Index: 191754 (Open Web Text)

Max Activating Token Index: 333

Text #14

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4494. Min Act: -0.1700

Data Index: 4333469 (Open Web Text)

Max Activating Token Index: 351

Text #15

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4347. Min Act: -0.1700

Data Index: 4210435 (Open Web Text)

Max Activating Token Index: 80

Text #16

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4376. Min Act: -0.1700

Data Index: 3200971 (Open Web Text)

Max Activating Token Index: 212

Text #17

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4008. Min Act: -0.1700

Data Index: 2095522 (Open Web Text)

Max Activating Token Index: 517

Text #18

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4004. Min Act: -0.1700

Data Index: 2312775 (Open Web Text)

Max Activating Token Index: 429

Text #19

Max Range: 4.4928. Min Range: -4.4928

Max Act: 4.4272. Min Act: -0.1700

Data Index: 2457334 (Open Web Text)

Max Activating Token Index: 498