Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1626 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.8235. Min Act: -0.1700

Data Index: 5537294 (Open Web Text)

Max Activating Token Index: 74

Text #1

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.7762. Min Act: -0.1700

Data Index: 3806092 (Open Web Text)

Max Activating Token Index: 257

Text #2

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.7092. Min Act: -0.1700

Data Index: 8463267 (Open Web Text)

Max Activating Token Index: 12

Text #3

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.7263. Min Act: -0.1700

Data Index: 179283 (Open Web Text)

Max Activating Token Index: 905

Text #4

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.7275. Min Act: -0.1700

Data Index: 5280935 (Open Web Text)

Max Activating Token Index: 727

Text #5

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.6467. Min Act: -0.1700

Data Index: 6241730 (Open Web Text)

Max Activating Token Index: 185

Text #6

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.6618. Min Act: -0.1700

Data Index: 7600778 (Open Web Text)

Max Activating Token Index: 235

Text #7

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.6358. Min Act: -0.1700

Data Index: 1747427 (Open Web Text)

Max Activating Token Index: 819

Text #8

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.6155. Min Act: -0.1700

Data Index: 4119318 (Open Web Text)

Max Activating Token Index: 906

Text #9

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.6613. Min Act: -0.1700

Data Index: 8222524 (Open Web Text)

Max Activating Token Index: 858

Text #10

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.5995. Min Act: -0.1700

Data Index: 7246527 (Open Web Text)

Max Activating Token Index: 701

Text #11

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.6457. Min Act: -0.1700

Data Index: 7988 (Open Web Text)

Max Activating Token Index: 738

Text #12

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.5859. Min Act: -0.1700

Data Index: 8667930 (Open Web Text)

Max Activating Token Index: 43

Text #13

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.6061. Min Act: -0.1700

Data Index: 8462679 (Open Web Text)

Max Activating Token Index: 649

Text #14

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.5965. Min Act: -0.1700

Data Index: 7146420 (Open Web Text)

Max Activating Token Index: 879

Text #15

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.6039. Min Act: -0.1700

Data Index: 5620805 (Open Web Text)

Max Activating Token Index: 448

Text #16

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.5631. Min Act: -0.1700

Data Index: 4762902 (Open Web Text)

Max Activating Token Index: 627

Text #17

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.5972. Min Act: -0.1700

Data Index: 5922470 (Open Web Text)

Max Activating Token Index: 952

Text #18

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.5651. Min Act: -0.1700

Data Index: 2221047 (Open Web Text)

Max Activating Token Index: 655

Text #19

Max Range: 3.8235. Min Range: -3.8235

Max Act: 3.5979. Min Act: -0.1700

Data Index: 1238831 (Open Web Text)

Max Activating Token Index: 13