Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1571. Min Range: -5.1571

Max Act: 5.1571. Min Act: -0.1700

Data Index: 5200694 (Open Web Text)

Max Activating Token Index: 163

Text #1

Max Range: 5.1571. Min Range: -5.1571

Max Act: 5.1382. Min Act: -0.1700

Data Index: 4654714 (Open Web Text)

Max Activating Token Index: 99

Text #2

Max Range: 5.1571. Min Range: -5.1571

Max Act: 5.0986. Min Act: -0.1700

Data Index: 3182525 (Open Web Text)

Max Activating Token Index: 788

Text #3

Max Range: 5.1571. Min Range: -5.1571

Max Act: 5.0663. Min Act: -0.1700

Data Index: 7509403 (Open Web Text)

Max Activating Token Index: 709

Text #4

Max Range: 5.1571. Min Range: -5.1571

Max Act: 5.0594. Min Act: -0.1700

Data Index: 6239095 (Open Web Text)

Max Activating Token Index: 821

Text #5

Max Range: 5.1571. Min Range: -5.1571

Max Act: 5.0428. Min Act: -0.1700

Data Index: 2550355 (Open Web Text)

Max Activating Token Index: 617

Text #6

Max Range: 5.1571. Min Range: -5.1571

Max Act: 5.0203. Min Act: -0.1700

Data Index: 6802515 (Open Web Text)

Max Activating Token Index: 813

Text #7

Max Range: 5.1571. Min Range: -5.1571

Max Act: 5.0220. Min Act: -0.1700

Data Index: 7703074 (Open Web Text)

Max Activating Token Index: 462

Text #8

Max Range: 5.1571. Min Range: -5.1571

Max Act: 4.9844. Min Act: -0.1700

Data Index: 4840438 (Open Web Text)

Max Activating Token Index: 521

Text #9

Max Range: 5.1571. Min Range: -5.1571

Max Act: 4.9812. Min Act: -0.1700

Data Index: 4301323 (Open Web Text)

Max Activating Token Index: 260

Text #10

Max Range: 5.1571. Min Range: -5.1571

Max Act: 4.9871. Min Act: -0.1700

Data Index: 4890173 (Open Web Text)

Max Activating Token Index: 823

Text #11

Max Range: 5.1571. Min Range: -5.1571

Max Act: 4.9826. Min Act: -0.1700

Data Index: 4103466 (Open Web Text)

Max Activating Token Index: 337

Text #12

Max Range: 5.1571. Min Range: -5.1571

Max Act: 5.0063. Min Act: -0.1700

Data Index: 6803722 (Open Web Text)

Max Activating Token Index: 264

Text #13

Max Range: 5.1571. Min Range: -5.1571

Max Act: 4.9970. Min Act: -0.1700

Data Index: 5830558 (Open Web Text)

Max Activating Token Index: 168

Text #14

Max Range: 5.1571. Min Range: -5.1571

Max Act: 4.9970. Min Act: -0.1700

Data Index: 2012116 (Open Web Text)

Max Activating Token Index: 940

Text #15

Max Range: 5.1571. Min Range: -5.1571

Max Act: 4.9566. Min Act: -0.1700

Data Index: 507607 (Open Web Text)

Max Activating Token Index: 971

Text #16

Max Range: 5.1571. Min Range: -5.1571

Max Act: 4.9469. Min Act: -0.1700

Data Index: 5331879 (Open Web Text)

Max Activating Token Index: 392

Text #17

Max Range: 5.1571. Min Range: -5.1571

Max Act: 4.9483. Min Act: -0.1700

Data Index: 5387450 (Open Web Text)

Max Activating Token Index: 296

Text #18

Max Range: 5.1571. Min Range: -5.1571

Max Act: 4.9550. Min Act: -0.1700

Data Index: 6024427 (Open Web Text)

Max Activating Token Index: 603

Text #19

Max Range: 5.1571. Min Range: -5.1571

Max Act: 4.9626. Min Act: -0.1700

Data Index: 1499387 (Open Web Text)

Max Activating Token Index: 865