Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1401 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.8094. Min Act: -0.1699

Data Index: 6932903 (Open Web Text)

Max Activating Token Index: 263

Text #1

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7677. Min Act: -0.1700

Data Index: 3656477 (Open Web Text)

Max Activating Token Index: 492

Text #2

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7692. Min Act: -0.1700

Data Index: 1140120 (Open Web Text)

Max Activating Token Index: 336

Text #3

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7713. Min Act: -0.1700

Data Index: 3239816 (Open Web Text)

Max Activating Token Index: 354

Text #4

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7611. Min Act: -0.1700

Data Index: 6748340 (Open Web Text)

Max Activating Token Index: 547

Text #5

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7598. Min Act: -0.1674

Data Index: 2481449 (Open Web Text)

Max Activating Token Index: 416

Text #6

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7738. Min Act: -0.1684

Data Index: 5583710 (Open Web Text)

Max Activating Token Index: 409

Text #7

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7588. Min Act: -0.1700

Data Index: 7737433 (Open Web Text)

Max Activating Token Index: 597

Text #8

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7666. Min Act: -0.1700

Data Index: 4153985 (Open Web Text)

Max Activating Token Index: 269

Text #9

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7538. Min Act: -0.1698

Data Index: 8829522 (Open Web Text)

Max Activating Token Index: 589

Text #10

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7483. Min Act: -0.1700

Data Index: 5692841 (Open Web Text)

Max Activating Token Index: 409

Text #11

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7398. Min Act: -0.1696

Data Index: 6385404 (Open Web Text)

Max Activating Token Index: 521

Text #12

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7427. Min Act: -0.1699

Data Index: 1215642 (Open Web Text)

Max Activating Token Index: 415

Text #13

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7398. Min Act: -0.1700

Data Index: 2458499 (Open Web Text)

Max Activating Token Index: 421

Text #14

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7464. Min Act: -0.1700

Data Index: 5427916 (Open Web Text)

Max Activating Token Index: 469

Text #15

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7452. Min Act: -0.1700

Data Index: 3653890 (Open Web Text)

Max Activating Token Index: 320

Text #16

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7357. Min Act: -0.1700

Data Index: 7315044 (Open Web Text)

Max Activating Token Index: 503

Text #17

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7229. Min Act: -0.1699

Data Index: 6332492 (Open Web Text)

Max Activating Token Index: 709

Text #18

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7286. Min Act: -0.1700

Data Index: 6021640 (Open Web Text)

Max Activating Token Index: 958

Text #19

Max Range: 1.8094. Min Range: -1.8094

Max Act: 1.7276. Min Act: -0.1700

Data Index: 6445900 (Open Web Text)

Max Activating Token Index: 299