Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 103 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0944. Min Range: -4.0944

Max Act: 4.0944. Min Act: -0.1700

Data Index: 5811270 (Open Web Text)

Max Activating Token Index: 953

Text #1

Max Range: 4.0944. Min Range: -4.0944

Max Act: 4.0591. Min Act: -0.1700

Data Index: 1079253 (Open Web Text)

Max Activating Token Index: 118

Text #2

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.9439. Min Act: -0.1700

Data Index: 8717708 (Open Web Text)

Max Activating Token Index: 1021

Text #3

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.9374. Min Act: -0.1700

Data Index: 3404263 (Open Web Text)

Max Activating Token Index: 285

Text #4

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.9490. Min Act: -0.1700

Data Index: 8484238 (Open Web Text)

Max Activating Token Index: 35

Text #5

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.9196. Min Act: -0.1700

Data Index: 5517128 (Open Web Text)

Max Activating Token Index: 800

Text #6

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.9211. Min Act: -0.1700

Data Index: 1022282 (Open Web Text)

Max Activating Token Index: 878

Text #7

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.9044. Min Act: -0.1700

Data Index: 2449247 (Open Web Text)

Max Activating Token Index: 475

Text #8

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.8833. Min Act: -0.1700

Data Index: 3743564 (Open Web Text)

Max Activating Token Index: 501

Text #9

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.7816. Min Act: -0.1700

Data Index: 4924743 (Open Web Text)

Max Activating Token Index: 1019

Text #10

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.8301. Min Act: -0.1700

Data Index: 8742810 (Open Web Text)

Max Activating Token Index: 981

Text #11

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.8254. Min Act: -0.1700

Data Index: 6355433 (Open Web Text)

Max Activating Token Index: 468

Text #12

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.8157. Min Act: -0.1700

Data Index: 2702468 (Open Web Text)

Max Activating Token Index: 609

Text #13

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.8230. Min Act: -0.1700

Data Index: 3888693 (Open Web Text)

Max Activating Token Index: 250

Text #14

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.8064. Min Act: -0.1700

Data Index: 1437548 (Open Web Text)

Max Activating Token Index: 839

Text #15

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.7615. Min Act: -0.1700

Data Index: 3699967 (Open Web Text)

Max Activating Token Index: 161

Text #16

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.7949. Min Act: -0.1700

Data Index: 8546700 (Open Web Text)

Max Activating Token Index: 502

Text #17

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.7612. Min Act: -0.1700

Data Index: 2018689 (Open Web Text)

Max Activating Token Index: 640

Text #18

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.7910. Min Act: -0.1700

Data Index: 3999778 (Open Web Text)

Max Activating Token Index: 36

Text #19

Max Range: 4.0944. Min Range: -4.0944

Max Act: 3.7669. Min Act: -0.1700

Data Index: 3100006 (Open Web Text)

Max Activating Token Index: 541