Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 338 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6779. Min Act: -0.1700

Data Index: 6268004 (Open Web Text)

Max Activating Token Index: 507

Text #1

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6562. Min Act: -0.1700

Data Index: 645542 (Open Web Text)

Max Activating Token Index: 629

Text #2

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6549. Min Act: -0.1700

Data Index: 3527846 (Open Web Text)

Max Activating Token Index: 644

Text #3

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6479. Min Act: -0.1700

Data Index: 8393118 (Open Web Text)

Max Activating Token Index: 207

Text #4

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6284. Min Act: -0.1700

Data Index: 6160202 (Open Web Text)

Max Activating Token Index: 8

Text #5

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6175. Min Act: -0.1700

Data Index: 208545 (Open Web Text)

Max Activating Token Index: 688

Text #6

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6255. Min Act: -0.1700

Data Index: 8181799 (Open Web Text)

Max Activating Token Index: 678

Text #7

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6334. Min Act: -0.1700

Data Index: 535367 (Open Web Text)

Max Activating Token Index: 224

Text #8

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6183. Min Act: -0.1700

Data Index: 3904038 (Open Web Text)

Max Activating Token Index: 154

Text #9

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6115. Min Act: -0.1700

Data Index: 7413521 (Open Web Text)

Max Activating Token Index: 564

Text #10

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.5990. Min Act: -0.1699

Data Index: 1011050 (Open Web Text)

Max Activating Token Index: 798

Text #11

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6053. Min Act: -0.1700

Data Index: 7175022 (Open Web Text)

Max Activating Token Index: 174

Text #12

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6060. Min Act: -0.1700

Data Index: 6445386 (Open Web Text)

Max Activating Token Index: 525

Text #13

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.5994. Min Act: -0.1700

Data Index: 6570946 (Open Web Text)

Max Activating Token Index: 705

Text #14

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.5991. Min Act: -0.1700

Data Index: 7438833 (Open Web Text)

Max Activating Token Index: 200

Text #15

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.5989. Min Act: -0.1700

Data Index: 2336385 (Open Web Text)

Max Activating Token Index: 640

Text #16

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.6036. Min Act: -0.1700

Data Index: 3807456 (Open Web Text)

Max Activating Token Index: 192

Text #17

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.5936. Min Act: -0.1696

Data Index: 4858033 (Open Web Text)

Max Activating Token Index: 192

Text #18

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.5963. Min Act: -0.1700

Data Index: 5102909 (Open Web Text)

Max Activating Token Index: 482

Text #19

Max Range: 1.6779. Min Range: -1.6779

Max Act: 1.5940. Min Act: -0.1699

Data Index: 1124355 (Open Web Text)

Max Activating Token Index: 733