Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1242 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.5115. Min Act: -0.1700

Data Index: 1961633 (Open Web Text)

Max Activating Token Index: 508

Text #1

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.4236. Min Act: -0.1688

Data Index: 3347004 (Open Web Text)

Max Activating Token Index: 476

Text #2

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3989. Min Act: -0.1700

Data Index: 479403 (Open Web Text)

Max Activating Token Index: 393

Text #3

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3784. Min Act: -0.1700

Data Index: 1278508 (Open Web Text)

Max Activating Token Index: 241

Text #4

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3733. Min Act: -0.1699

Data Index: 2345592 (Open Web Text)

Max Activating Token Index: 575

Text #5

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3778. Min Act: -0.1700

Data Index: 5125077 (Open Web Text)

Max Activating Token Index: 823

Text #6

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3556. Min Act: -0.1700

Data Index: 2153293 (Open Web Text)

Max Activating Token Index: 870

Text #7

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3634. Min Act: -0.1700

Data Index: 705856 (Open Web Text)

Max Activating Token Index: 533

Text #8

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3674. Min Act: -0.1697

Data Index: 4451052 (Open Web Text)

Max Activating Token Index: 676

Text #9

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3606. Min Act: -0.1699

Data Index: 1173799 (Open Web Text)

Max Activating Token Index: 577

Text #10

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3552. Min Act: -0.1700

Data Index: 6931443 (Open Web Text)

Max Activating Token Index: 605

Text #11

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3541. Min Act: -0.1699

Data Index: 734451 (Open Web Text)

Max Activating Token Index: 573

Text #12

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3567. Min Act: -0.1677

Data Index: 709732 (Open Web Text)

Max Activating Token Index: 574

Text #13

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3498. Min Act: -0.1700

Data Index: 8265444 (Open Web Text)

Max Activating Token Index: 460

Text #14

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3439. Min Act: -0.1700

Data Index: 2599442 (Open Web Text)

Max Activating Token Index: 99

Text #15

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3479. Min Act: -0.1700

Data Index: 8777536 (Open Web Text)

Max Activating Token Index: 593

Text #16

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3418. Min Act: -0.1700

Data Index: 3547052 (Open Web Text)

Max Activating Token Index: 1020

Text #17

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3358. Min Act: -0.1700

Data Index: 7607471 (Open Web Text)

Max Activating Token Index: 516

Text #18

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3326. Min Act: -0.1700

Data Index: 81228 (Open Web Text)

Max Activating Token Index: 472

Text #19

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3381. Min Act: -0.1686

Data Index: 6356210 (Open Web Text)

Max Activating Token Index: 424