Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1061 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1945. Min Act: -0.1700

Data Index: 5635359 (Open Web Text)

Max Activating Token Index: 507

Text #1

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1353. Min Act: -0.1700

Data Index: 7218097 (Open Web Text)

Max Activating Token Index: 409

Text #2

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1289. Min Act: -0.1700

Data Index: 818992 (Open Web Text)

Max Activating Token Index: 444

Text #3

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1204. Min Act: -0.1700

Data Index: 4731193 (Open Web Text)

Max Activating Token Index: 780

Text #4

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1484. Min Act: -0.1700

Data Index: 1350819 (Open Web Text)

Max Activating Token Index: 1011

Text #5

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1174. Min Act: -0.1700

Data Index: 1590470 (Open Web Text)

Max Activating Token Index: 135

Text #6

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1026. Min Act: -0.1700

Data Index: 3524328 (Open Web Text)

Max Activating Token Index: 863

Text #7

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1080. Min Act: -0.1700

Data Index: 7081134 (Open Web Text)

Max Activating Token Index: 385

Text #8

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0886. Min Act: -0.1700

Data Index: 7723486 (Open Web Text)

Max Activating Token Index: 236

Text #9

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1019. Min Act: -0.1700

Data Index: 3217805 (Open Web Text)

Max Activating Token Index: 354

Text #10

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0787. Min Act: -0.1700

Data Index: 6383313 (Open Web Text)

Max Activating Token Index: 6

Text #11

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0711. Min Act: -0.1700

Data Index: 2107151 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0711. Min Act: -0.1700

Data Index: 2595121 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0454. Min Act: -0.1700

Data Index: 8770570 (Open Web Text)

Max Activating Token Index: 637

Text #14

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0785. Min Act: -0.1700

Data Index: 1714703 (Open Web Text)

Max Activating Token Index: 644

Text #15

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0729. Min Act: -0.1700

Data Index: 8272160 (Open Web Text)

Max Activating Token Index: 997

Text #16

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0381. Min Act: -0.1700

Data Index: 234541 (Open Web Text)

Max Activating Token Index: 438

Text #17

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0551. Min Act: -0.1700

Data Index: 7773519 (Open Web Text)

Max Activating Token Index: 393

Text #18

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0493. Min Act: -0.1700

Data Index: 4516195 (Open Web Text)

Max Activating Token Index: 813

Text #19

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0319. Min Act: -0.1700

Data Index: 6806216 (Open Web Text)

Max Activating Token Index: 523