Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2046 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0580. Min Range: -3.0580

Max Act: 3.0580. Min Act: -0.1700

Data Index: 4168460 (Open Web Text)

Max Activating Token Index: 674

Text #1

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9949. Min Act: -0.1700

Data Index: 4128764 (Open Web Text)

Max Activating Token Index: 460

Text #2

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9563. Min Act: -0.1700

Data Index: 599811 (Open Web Text)

Max Activating Token Index: 326

Text #3

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9427. Min Act: -0.1700

Data Index: 6964859 (Open Web Text)

Max Activating Token Index: 22

Text #4

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9268. Min Act: -0.1700

Data Index: 3039493 (Open Web Text)

Max Activating Token Index: 255

Text #5

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9335. Min Act: -0.1700

Data Index: 7521943 (Open Web Text)

Max Activating Token Index: 517

Text #6

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9274. Min Act: -0.1700

Data Index: 7810930 (Open Web Text)

Max Activating Token Index: 607

Text #7

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9302. Min Act: -0.1700

Data Index: 8277075 (Open Web Text)

Max Activating Token Index: 566

Text #8

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9353. Min Act: -0.1700

Data Index: 7438171 (Open Web Text)

Max Activating Token Index: 344

Text #9

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9344. Min Act: -0.1700

Data Index: 4128765 (Open Web Text)

Max Activating Token Index: 684

Text #10

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9196. Min Act: -0.1700

Data Index: 1692584 (Open Web Text)

Max Activating Token Index: 227

Text #11

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9183. Min Act: -0.1700

Data Index: 1032962 (Open Web Text)

Max Activating Token Index: 150

Text #12

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9259. Min Act: -0.1700

Data Index: 5512480 (Open Web Text)

Max Activating Token Index: 680

Text #13

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9306. Min Act: -0.1700

Data Index: 4634386 (Open Web Text)

Max Activating Token Index: 621

Text #14

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9229. Min Act: -0.1700

Data Index: 3512539 (Open Web Text)

Max Activating Token Index: 604

Text #15

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9145. Min Act: -0.1700

Data Index: 1635006 (Open Web Text)

Max Activating Token Index: 573

Text #16

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9350. Min Act: -0.1700

Data Index: 2002274 (Open Web Text)

Max Activating Token Index: 202

Text #17

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9103. Min Act: -0.1700

Data Index: 1782282 (Open Web Text)

Max Activating Token Index: 720

Text #18

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9008. Min Act: -0.1700

Data Index: 5757330 (Open Web Text)

Max Activating Token Index: 252

Text #19

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9040. Min Act: -0.1700

Data Index: 1621188 (Open Web Text)

Max Activating Token Index: 638