Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 596 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.5362. Min Act: -0.1700

Data Index: 508969 (Open Web Text)

Max Activating Token Index: 985

Text #1

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.4645. Min Act: -0.1700

Data Index: 7613329 (Open Web Text)

Max Activating Token Index: 667

Text #2

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.4454. Min Act: -0.1700

Data Index: 7215456 (Open Web Text)

Max Activating Token Index: 163

Text #3

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3834. Min Act: -0.1700

Data Index: 2734296 (Open Web Text)

Max Activating Token Index: 526

Text #4

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3899. Min Act: -0.1700

Data Index: 4239213 (Open Web Text)

Max Activating Token Index: 813

Text #5

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3873. Min Act: -0.1700

Data Index: 7032997 (Open Web Text)

Max Activating Token Index: 79

Text #6

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3899. Min Act: -0.1700

Data Index: 3047643 (Open Web Text)

Max Activating Token Index: 577

Text #7

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3554. Min Act: -0.1700

Data Index: 5738273 (Open Web Text)

Max Activating Token Index: 635

Text #8

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3428. Min Act: -0.1700

Data Index: 2939721 (Open Web Text)

Max Activating Token Index: 848

Text #9

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3327. Min Act: -0.1700

Data Index: 7678674 (Open Web Text)

Max Activating Token Index: 680

Text #10

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3714. Min Act: -0.1700

Data Index: 4325655 (Open Web Text)

Max Activating Token Index: 557

Text #11

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3640. Min Act: -0.1700

Data Index: 1932956 (Open Web Text)

Max Activating Token Index: 965

Text #12

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3708. Min Act: -0.1700

Data Index: 3753270 (Open Web Text)

Max Activating Token Index: 63

Text #13

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3416. Min Act: -0.1700

Data Index: 1706707 (Open Web Text)

Max Activating Token Index: 200

Text #14

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3444. Min Act: -0.1700

Data Index: 277920 (Open Web Text)

Max Activating Token Index: 167

Text #15

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3538. Min Act: -0.1700

Data Index: 6472524 (Open Web Text)

Max Activating Token Index: 681

Text #16

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3297. Min Act: -0.1700

Data Index: 6288753 (Open Web Text)

Max Activating Token Index: 168

Text #17

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3578. Min Act: -0.1700

Data Index: 1359436 (Open Web Text)

Max Activating Token Index: 661

Text #18

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3625. Min Act: -0.1700

Data Index: 5518506 (Open Web Text)

Max Activating Token Index: 246

Text #19

Max Range: 4.5362. Min Range: -4.5362

Max Act: 4.3142. Min Act: -0.1700

Data Index: 5045687 (Open Web Text)

Max Activating Token Index: 441