Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2026 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.5447. Min Act: -0.1287

Data Index: 6163618 (Open Web Text)

Max Activating Token Index: 619

Text #1

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.5133. Min Act: -0.1556

Data Index: 5870967 (Open Web Text)

Max Activating Token Index: 63

Text #2

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.5033. Min Act: -0.1055

Data Index: 581283 (Open Web Text)

Max Activating Token Index: 489

Text #3

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4922. Min Act: -0.1329

Data Index: 5875537 (Open Web Text)

Max Activating Token Index: 146

Text #4

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4886. Min Act: -0.1585

Data Index: 3475203 (Open Web Text)

Max Activating Token Index: 162

Text #5

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4944. Min Act: -0.0979

Data Index: 2857779 (Open Web Text)

Max Activating Token Index: 646

Text #6

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4771. Min Act: -0.1379

Data Index: 3075111 (Open Web Text)

Max Activating Token Index: 383

Text #7

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4750. Min Act: -0.1284

Data Index: 5406814 (Open Web Text)

Max Activating Token Index: 251

Text #8

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4713. Min Act: -0.0980

Data Index: 1462961 (Open Web Text)

Max Activating Token Index: 847

Text #9

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4772. Min Act: -0.1056

Data Index: 256520 (Open Web Text)

Max Activating Token Index: 730

Text #10

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4815. Min Act: -0.1297

Data Index: 1461002 (Open Web Text)

Max Activating Token Index: 23

Text #11

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4697. Min Act: -0.1312

Data Index: 8387454 (Open Web Text)

Max Activating Token Index: 862

Text #12

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4649. Min Act: -0.0903

Data Index: 5373760 (Open Web Text)

Max Activating Token Index: 673

Text #13

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4577. Min Act: -0.1673

Data Index: 1902981 (Open Web Text)

Max Activating Token Index: 366

Text #14

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4713. Min Act: -0.1582

Data Index: 3626876 (Open Web Text)

Max Activating Token Index: 226

Text #15

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4608. Min Act: -0.1245

Data Index: 7778872 (Open Web Text)

Max Activating Token Index: 432

Text #16

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4627. Min Act: -0.1272

Data Index: 1031893 (Open Web Text)

Max Activating Token Index: 170

Text #17

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4660. Min Act: -0.1211

Data Index: 8475989 (Open Web Text)

Max Activating Token Index: 883

Text #18

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4620. Min Act: -0.1396

Data Index: 7869262 (Open Web Text)

Max Activating Token Index: 588

Text #19

Max Range: 1.5447. Min Range: -1.5447

Max Act: 1.4554. Min Act: -0.1391

Data Index: 4836880 (Open Web Text)

Max Activating Token Index: 415