Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 890 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.1185. Min Act: -0.1253

Data Index: 1139437 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.1242. Min Act: -0.1394

Data Index: 8377991 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.1219. Min Act: -0.1236

Data Index: 6139313 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.1131. Min Act: -0.1273

Data Index: 2459332 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0995. Min Act: -0.1071

Data Index: 1139438 (Open Web Text)

Max Activating Token Index: 673

Text #5

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.1129. Min Act: -0.1481

Data Index: 614554 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0799. Min Act: -0.1293

Data Index: 5629861 (Open Web Text)

Max Activating Token Index: 601

Text #7

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0809. Min Act: -0.1267

Data Index: 1139439 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0775. Min Act: -0.1213

Data Index: 7822537 (Open Web Text)

Max Activating Token Index: 127

Text #9

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0690. Min Act: -0.1443

Data Index: 6983993 (Open Web Text)

Max Activating Token Index: 709

Text #10

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0722. Min Act: -0.1646

Data Index: 870301 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0722. Min Act: -0.1505

Data Index: 4967949 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0670. Min Act: -0.1299

Data Index: 6891951 (Open Web Text)

Max Activating Token Index: 485

Text #13

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0568. Min Act: -0.0267

Data Index: 1454635 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0509. Min Act: -0.1258

Data Index: 1589403 (Open Web Text)

Max Activating Token Index: 18

Text #15

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0554. Min Act: -0.1489

Data Index: 6702541 (Open Web Text)

Max Activating Token Index: 73

Text #16

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0452. Min Act: -0.1610

Data Index: 3645234 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0411. Min Act: -0.1536

Data Index: 3724283 (Open Web Text)

Max Activating Token Index: 9

Text #18

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0388. Min Act: -0.1226

Data Index: 436397 (Open Web Text)

Max Activating Token Index: 687

Text #19

Max Range: 1.1242. Min Range: -1.1242

Max Act: 1.0444. Min Act: -0.1193

Data Index: 5886691 (Open Web Text)

Max Activating Token Index: 4