Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1737 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.5651. Min Act: -0.1700

Data Index: 2625669 (Open Web Text)

Max Activating Token Index: 927

Text #1

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.5391. Min Act: -0.1700

Data Index: 8074790 (Open Web Text)

Max Activating Token Index: 762

Text #2

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.5141. Min Act: -0.1700

Data Index: 2790619 (Open Web Text)

Max Activating Token Index: 777

Text #3

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4950. Min Act: -0.1700

Data Index: 4012682 (Open Web Text)

Max Activating Token Index: 895

Text #4

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.5093. Min Act: -0.1700

Data Index: 65731 (Open Web Text)

Max Activating Token Index: 488

Text #5

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4407. Min Act: -0.1700

Data Index: 595306 (Open Web Text)

Max Activating Token Index: 493

Text #6

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4615. Min Act: -0.1700

Data Index: 4615447 (Open Web Text)

Max Activating Token Index: 694

Text #7

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4498. Min Act: -0.1700

Data Index: 8129107 (Open Web Text)

Max Activating Token Index: 687

Text #8

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4660. Min Act: -0.1700

Data Index: 2366833 (Open Web Text)

Max Activating Token Index: 264

Text #9

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4360. Min Act: -0.1700

Data Index: 5083025 (Open Web Text)

Max Activating Token Index: 744

Text #10

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4121. Min Act: -0.1700

Data Index: 3737146 (Open Web Text)

Max Activating Token Index: 1003

Text #11

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4115. Min Act: -0.1700

Data Index: 3415446 (Open Web Text)

Max Activating Token Index: 20

Text #12

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4530. Min Act: -0.1700

Data Index: 8567762 (Open Web Text)

Max Activating Token Index: 948

Text #13

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4198. Min Act: -0.1700

Data Index: 2461150 (Open Web Text)

Max Activating Token Index: 384

Text #14

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4408. Min Act: -0.1700

Data Index: 3038973 (Open Web Text)

Max Activating Token Index: 482

Text #15

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4320. Min Act: -0.1700

Data Index: 280015 (Open Web Text)

Max Activating Token Index: 279

Text #16

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.4039. Min Act: -0.1700

Data Index: 6233036 (Open Web Text)

Max Activating Token Index: 313

Text #17

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.3937. Min Act: -0.1700

Data Index: 8792542 (Open Web Text)

Max Activating Token Index: 167

Text #18

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.3941. Min Act: -0.1700

Data Index: 4660412 (Open Web Text)

Max Activating Token Index: 712

Text #19

Max Range: 4.5651. Min Range: -4.5651

Max Act: 4.3850. Min Act: -0.1700

Data Index: 5570626 (Open Web Text)

Max Activating Token Index: 640