Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 225 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.9145. Min Act: -0.1700

Data Index: 7851243 (Open Web Text)

Max Activating Token Index: 215

Text #1

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.8484. Min Act: -0.1700

Data Index: 1582090 (Open Web Text)

Max Activating Token Index: 398

Text #2

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.8103. Min Act: -0.1700

Data Index: 2426962 (Open Web Text)

Max Activating Token Index: 657

Text #3

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.8086. Min Act: -0.1700

Data Index: 4893528 (Open Web Text)

Max Activating Token Index: 894

Text #4

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7874. Min Act: -0.1700

Data Index: 8161720 (Open Web Text)

Max Activating Token Index: 210

Text #5

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7834. Min Act: -0.1700

Data Index: 8300394 (Open Web Text)

Max Activating Token Index: 268

Text #6

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7668. Min Act: -0.1700

Data Index: 40085 (Open Web Text)

Max Activating Token Index: 537

Text #7

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7816. Min Act: -0.1700

Data Index: 3138344 (Open Web Text)

Max Activating Token Index: 1021

Text #8

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7724. Min Act: -0.1700

Data Index: 5361750 (Open Web Text)

Max Activating Token Index: 612

Text #9

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7505. Min Act: -0.1700

Data Index: 870709 (Open Web Text)

Max Activating Token Index: 290

Text #10

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7403. Min Act: -0.1700

Data Index: 6690855 (Open Web Text)

Max Activating Token Index: 120

Text #11

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7678. Min Act: -0.1700

Data Index: 6473477 (Open Web Text)

Max Activating Token Index: 606

Text #12

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7466. Min Act: -0.1700

Data Index: 5840704 (Open Web Text)

Max Activating Token Index: 233

Text #13

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7594. Min Act: -0.1700

Data Index: 3208114 (Open Web Text)

Max Activating Token Index: 669

Text #14

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7482. Min Act: -0.1700

Data Index: 4850742 (Open Web Text)

Max Activating Token Index: 137

Text #15

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7498. Min Act: -0.1700

Data Index: 2317938 (Open Web Text)

Max Activating Token Index: 1020

Text #16

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7349. Min Act: -0.1700

Data Index: 468994 (Open Web Text)

Max Activating Token Index: 897

Text #17

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7409. Min Act: -0.1700

Data Index: 1509171 (Open Web Text)

Max Activating Token Index: 972

Text #18

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7417. Min Act: -0.1700

Data Index: 4724267 (Open Web Text)

Max Activating Token Index: 528

Text #19

Max Range: 3.9145. Min Range: -3.9145

Max Act: 3.7611. Min Act: -0.1700

Data Index: 1402856 (Open Web Text)

Max Activating Token Index: 453