Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 915 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.5311. Min Act: -0.1700

Data Index: 4238212 (Open Web Text)

Max Activating Token Index: 265

Text #1

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.5108. Min Act: -0.1700

Data Index: 7447815 (Open Web Text)

Max Activating Token Index: 210

Text #2

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.4797. Min Act: -0.1700

Data Index: 7037160 (Open Web Text)

Max Activating Token Index: 57

Text #3

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.4479. Min Act: -0.1700

Data Index: 4213149 (Open Web Text)

Max Activating Token Index: 83

Text #4

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.4434. Min Act: -0.1700

Data Index: 5144708 (Open Web Text)

Max Activating Token Index: 808

Text #5

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.4325. Min Act: -0.1700

Data Index: 8697333 (Open Web Text)

Max Activating Token Index: 783

Text #6

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.4278. Min Act: -0.1700

Data Index: 8454361 (Open Web Text)

Max Activating Token Index: 149

Text #7

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3761. Min Act: -0.1700

Data Index: 3960413 (Open Web Text)

Max Activating Token Index: 863

Text #8

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3526. Min Act: -0.1700

Data Index: 4167787 (Open Web Text)

Max Activating Token Index: 58

Text #9

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3682. Min Act: -0.1700

Data Index: 3323667 (Open Web Text)

Max Activating Token Index: 370

Text #10

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3670. Min Act: -0.1700

Data Index: 7921523 (Open Web Text)

Max Activating Token Index: 416

Text #11

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3517. Min Act: -0.1700

Data Index: 5278317 (Open Web Text)

Max Activating Token Index: 25

Text #12

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3631. Min Act: -0.1700

Data Index: 2160200 (Open Web Text)

Max Activating Token Index: 160

Text #13

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3682. Min Act: -0.1700

Data Index: 7751315 (Open Web Text)

Max Activating Token Index: 240

Text #14

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3445. Min Act: -0.1700

Data Index: 3594215 (Open Web Text)

Max Activating Token Index: 209

Text #15

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3895. Min Act: -0.1700

Data Index: 6511702 (Open Web Text)

Max Activating Token Index: 620

Text #16

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3658. Min Act: -0.1700

Data Index: 4517035 (Open Web Text)

Max Activating Token Index: 692

Text #17

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3535. Min Act: -0.1700

Data Index: 8216799 (Open Web Text)

Max Activating Token Index: 447

Text #18

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3428. Min Act: -0.1700

Data Index: 1295741 (Open Web Text)

Max Activating Token Index: 206

Text #19

Max Range: 3.5311. Min Range: -3.5311

Max Act: 3.3438. Min Act: -0.1700

Data Index: 8270615 (Open Web Text)

Max Activating Token Index: 324