Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 658 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.5115. Min Act: -0.1699

Data Index: 824485 (Open Web Text)

Max Activating Token Index: 505

Text #1

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.4845. Min Act: -0.1694

Data Index: 6061575 (Open Web Text)

Max Activating Token Index: 602

Text #2

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.4666. Min Act: -0.1700

Data Index: 559287 (Open Web Text)

Max Activating Token Index: 60

Text #3

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.4575. Min Act: -0.1678

Data Index: 8574901 (Open Web Text)

Max Activating Token Index: 448

Text #4

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.4549. Min Act: -0.1699

Data Index: 4229779 (Open Web Text)

Max Activating Token Index: 240

Text #5

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.4397. Min Act: -0.1697

Data Index: 1887569 (Open Web Text)

Max Activating Token Index: 89

Text #6

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.4367. Min Act: -0.1691

Data Index: 1910201 (Open Web Text)

Max Activating Token Index: 178

Text #7

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.4253. Min Act: -0.1700

Data Index: 4086888 (Open Web Text)

Max Activating Token Index: 618

Text #8

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.4122. Min Act: -0.1699

Data Index: 7999235 (Open Web Text)

Max Activating Token Index: 925

Text #9

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3932. Min Act: -0.1699

Data Index: 332985 (Open Web Text)

Max Activating Token Index: 953

Text #10

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3962. Min Act: -0.1698

Data Index: 3407466 (Open Web Text)

Max Activating Token Index: 662

Text #11

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3948. Min Act: -0.1700

Data Index: 3679802 (Open Web Text)

Max Activating Token Index: 986

Text #12

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3918. Min Act: -0.1685

Data Index: 4406406 (Open Web Text)

Max Activating Token Index: 236

Text #13

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3815. Min Act: -0.1699

Data Index: 3476949 (Open Web Text)

Max Activating Token Index: 336

Text #14

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3704. Min Act: -0.1700

Data Index: 2889500 (Open Web Text)

Max Activating Token Index: 240

Text #15

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3748. Min Act: -0.1699

Data Index: 5537559 (Open Web Text)

Max Activating Token Index: 592

Text #16

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3710. Min Act: -0.1700

Data Index: 7556891 (Open Web Text)

Max Activating Token Index: 666

Text #17

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3708. Min Act: -0.1700

Data Index: 2460653 (Open Web Text)

Max Activating Token Index: 538

Text #18

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3607. Min Act: -0.1691

Data Index: 7482239 (Open Web Text)

Max Activating Token Index: 753

Text #19

Max Range: 1.5115. Min Range: -1.5115

Max Act: 1.3491. Min Act: -0.1688

Data Index: 3352892 (Open Web Text)

Max Activating Token Index: 924