Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 454 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.7732. Min Act: -0.1700

Data Index: 5477369 (Open Web Text)

Max Activating Token Index: 108

Text #1

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.7544. Min Act: -0.1700

Data Index: 3815052 (Open Web Text)

Max Activating Token Index: 952

Text #2

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.7450. Min Act: -0.1700

Data Index: 232994 (Open Web Text)

Max Activating Token Index: 10

Text #3

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.7502. Min Act: -0.1700

Data Index: 1823413 (Open Web Text)

Max Activating Token Index: 331

Text #4

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.7242. Min Act: -0.1700

Data Index: 7200346 (Open Web Text)

Max Activating Token Index: 199

Text #5

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.7065. Min Act: -0.1700

Data Index: 6660889 (Open Web Text)

Max Activating Token Index: 773

Text #6

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6980. Min Act: -0.1700

Data Index: 8295632 (Open Web Text)

Max Activating Token Index: 20

Text #7

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6649. Min Act: -0.1700

Data Index: 1949278 (Open Web Text)

Max Activating Token Index: 675

Text #8

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6704. Min Act: -0.1700

Data Index: 4507361 (Open Web Text)

Max Activating Token Index: 533

Text #9

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6676. Min Act: -0.1700

Data Index: 1709501 (Open Web Text)

Max Activating Token Index: 652

Text #10

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6932. Min Act: -0.1700

Data Index: 3855737 (Open Web Text)

Max Activating Token Index: 948

Text #11

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6533. Min Act: -0.1700

Data Index: 8257988 (Open Web Text)

Max Activating Token Index: 534

Text #12

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6925. Min Act: -0.1700

Data Index: 4172463 (Open Web Text)

Max Activating Token Index: 1003

Text #13

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6415. Min Act: -0.1700

Data Index: 4729076 (Open Web Text)

Max Activating Token Index: 873

Text #14

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6380. Min Act: -0.1700

Data Index: 8814863 (Open Web Text)

Max Activating Token Index: 571

Text #15

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6435. Min Act: -0.1700

Data Index: 519845 (Open Web Text)

Max Activating Token Index: 432

Text #16

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6624. Min Act: -0.1700

Data Index: 6945116 (Open Web Text)

Max Activating Token Index: 642

Text #17

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6335. Min Act: -0.1700

Data Index: 3745174 (Open Web Text)

Max Activating Token Index: 124

Text #18

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6107. Min Act: -0.1700

Data Index: 3151279 (Open Web Text)

Max Activating Token Index: 141

Text #19

Max Range: 3.7732. Min Range: -3.7732

Max Act: 3.6134. Min Act: -0.1700

Data Index: 5340496 (Open Web Text)

Max Activating Token Index: 9