Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2690 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.6267. Min Act: -0.1356

Data Index: 1948012 (Open Web Text)

Max Activating Token Index: 669

Text #1

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.6083. Min Act: -0.1502

Data Index: 4724144 (Open Web Text)

Max Activating Token Index: 485

Text #2

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5829. Min Act: -0.1423

Data Index: 4724371 (Open Web Text)

Max Activating Token Index: 448

Text #3

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5557. Min Act: -0.1601

Data Index: 3797179 (Open Web Text)

Max Activating Token Index: 645

Text #4

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5682. Min Act: -0.1276

Data Index: 4311407 (Open Web Text)

Max Activating Token Index: 545

Text #5

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5672. Min Act: -0.1644

Data Index: 1282421 (Open Web Text)

Max Activating Token Index: 216

Text #6

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5612. Min Act: -0.1560

Data Index: 2938510 (Open Web Text)

Max Activating Token Index: 127

Text #7

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5569. Min Act: -0.1646

Data Index: 3843970 (Open Web Text)

Max Activating Token Index: 78

Text #8

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5573. Min Act: -0.1685

Data Index: 7708159 (Open Web Text)

Max Activating Token Index: 771

Text #9

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5434. Min Act: -0.1219

Data Index: 5187824 (Open Web Text)

Max Activating Token Index: 728

Text #10

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5396. Min Act: -0.1655

Data Index: 5520012 (Open Web Text)

Max Activating Token Index: 378

Text #11

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5422. Min Act: -0.1014

Data Index: 5851185 (Open Web Text)

Max Activating Token Index: 798

Text #12

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5386. Min Act: -0.1423

Data Index: 4657516 (Open Web Text)

Max Activating Token Index: 38

Text #13

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5423. Min Act: -0.1662

Data Index: 2596644 (Open Web Text)

Max Activating Token Index: 38

Text #14

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5527. Min Act: -0.1578

Data Index: 7618403 (Open Web Text)

Max Activating Token Index: 898

Text #15

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5473. Min Act: -0.1496

Data Index: 120029 (Open Web Text)

Max Activating Token Index: 709

Text #16

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5353. Min Act: -0.1460

Data Index: 6485353 (Open Web Text)

Max Activating Token Index: 701

Text #17

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5440. Min Act: -0.1568

Data Index: 6885499 (Open Web Text)

Max Activating Token Index: 859

Text #18

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5303. Min Act: -0.1464

Data Index: 7622448 (Open Web Text)

Max Activating Token Index: 951

Text #19

Max Range: 1.6267. Min Range: -1.6267

Max Act: 1.5389. Min Act: -0.1619

Data Index: 2135409 (Open Web Text)

Max Activating Token Index: 704