Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 234 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5776. Min Act: -0.1700

Data Index: 3948910 (Open Web Text)

Max Activating Token Index: 526

Text #1

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5721. Min Act: -0.1700

Data Index: 8183249 (Open Web Text)

Max Activating Token Index: 48

Text #2

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5633. Min Act: -0.1700

Data Index: 7551432 (Open Web Text)

Max Activating Token Index: 426

Text #3

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5605. Min Act: -0.1700

Data Index: 4854778 (Open Web Text)

Max Activating Token Index: 206

Text #4

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5598. Min Act: -0.1700

Data Index: 2001427 (Open Web Text)

Max Activating Token Index: 78

Text #5

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5408. Min Act: -0.1700

Data Index: 177226 (Open Web Text)

Max Activating Token Index: 748

Text #6

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5472. Min Act: -0.1700

Data Index: 4367981 (Open Web Text)

Max Activating Token Index: 503

Text #7

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5415. Min Act: -0.1700

Data Index: 8667187 (Open Web Text)

Max Activating Token Index: 348

Text #8

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5420. Min Act: -0.1700

Data Index: 5395070 (Open Web Text)

Max Activating Token Index: 496

Text #9

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5442. Min Act: -0.1700

Data Index: 6476881 (Open Web Text)

Max Activating Token Index: 293

Text #10

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5462. Min Act: -0.1700

Data Index: 7324205 (Open Web Text)

Max Activating Token Index: 194

Text #11

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5471. Min Act: -0.1700

Data Index: 3765069 (Open Web Text)

Max Activating Token Index: 955

Text #12

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5279. Min Act: -0.1700

Data Index: 570366 (Open Web Text)

Max Activating Token Index: 291

Text #13

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5321. Min Act: -0.1700

Data Index: 1789865 (Open Web Text)

Max Activating Token Index: 248

Text #14

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5300. Min Act: -0.1700

Data Index: 4226554 (Open Web Text)

Max Activating Token Index: 484

Text #15

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5321. Min Act: -0.1700

Data Index: 5644328 (Open Web Text)

Max Activating Token Index: 221

Text #16

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5378. Min Act: -0.1700

Data Index: 6471210 (Open Web Text)

Max Activating Token Index: 102

Text #17

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5280. Min Act: -0.1700

Data Index: 6865677 (Open Web Text)

Max Activating Token Index: 246

Text #18

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5244. Min Act: -0.1700

Data Index: 734575 (Open Web Text)

Max Activating Token Index: 168

Text #19

Max Range: 3.5776. Min Range: -3.5776

Max Act: 3.5292. Min Act: -0.1700

Data Index: 3927451 (Open Web Text)

Max Activating Token Index: 450