Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2229 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4822. Min Act: -0.1568

Data Index: 7150096 (Open Web Text)

Max Activating Token Index: 564

Text #1

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4651. Min Act: -0.1588

Data Index: 2780791 (Open Web Text)

Max Activating Token Index: 8

Text #2

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4549. Min Act: -0.1686

Data Index: 7670857 (Open Web Text)

Max Activating Token Index: 514

Text #3

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4592. Min Act: -0.1681

Data Index: 2582367 (Open Web Text)

Max Activating Token Index: 984

Text #4

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4506. Min Act: -0.1679

Data Index: 1047689 (Open Web Text)

Max Activating Token Index: 7

Text #5

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4478. Min Act: -0.1635

Data Index: 6424639 (Open Web Text)

Max Activating Token Index: 11

Text #6

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4376. Min Act: -0.1438

Data Index: 8003593 (Open Web Text)

Max Activating Token Index: 7

Text #7

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4226. Min Act: -0.1585

Data Index: 1917565 (Open Web Text)

Max Activating Token Index: 184

Text #8

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4120. Min Act: -0.1601

Data Index: 8687419 (Open Web Text)

Max Activating Token Index: 12

Text #9

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4150. Min Act: -0.1699

Data Index: 4537081 (Open Web Text)

Max Activating Token Index: 885

Text #10

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4091. Min Act: -0.1692

Data Index: 5677909 (Open Web Text)

Max Activating Token Index: 258

Text #11

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4184. Min Act: -0.1680

Data Index: 3219745 (Open Web Text)

Max Activating Token Index: 688

Text #12

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4063. Min Act: -0.1243

Data Index: 7345167 (Open Web Text)

Max Activating Token Index: 155

Text #13

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4117. Min Act: -0.1673

Data Index: 6292179 (Open Web Text)

Max Activating Token Index: 900

Text #14

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.4045. Min Act: -0.1633

Data Index: 7149674 (Open Web Text)

Max Activating Token Index: 6

Text #15

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.3896. Min Act: -0.1505

Data Index: 7239523 (Open Web Text)

Max Activating Token Index: 720

Text #16

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.3912. Min Act: -0.1581

Data Index: 6056384 (Open Web Text)

Max Activating Token Index: 272

Text #17

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.3877. Min Act: -0.1700

Data Index: 4569027 (Open Web Text)

Max Activating Token Index: 752

Text #18

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.3778. Min Act: -0.1677

Data Index: 993570 (Open Web Text)

Max Activating Token Index: 14

Text #19

Max Range: 1.4822. Min Range: -1.4822

Max Act: 1.3762. Min Act: -0.1484

Data Index: 7444507 (Open Web Text)

Max Activating Token Index: 634