Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 434 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.9098. Min Act: -0.0747

Data Index: 2976894 (Open Web Text)

Max Activating Token Index: 599

Text #1

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.9056. Min Act: -0.1008

Data Index: 2976893 (Open Web Text)

Max Activating Token Index: 460

Text #2

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.8275. Min Act: -0.0747

Data Index: 5755455 (Open Web Text)

Max Activating Token Index: 299

Text #3

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.8315. Min Act: -0.1675

Data Index: 6136565 (Open Web Text)

Max Activating Token Index: 645

Text #4

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.8216. Min Act: -0.1076

Data Index: 5904501 (Open Web Text)

Max Activating Token Index: 637

Text #5

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7800. Min Act: -0.0747

Data Index: 5187443 (Open Web Text)

Max Activating Token Index: 885

Text #6

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7816. Min Act: -0.1112

Data Index: 4697465 (Open Web Text)

Max Activating Token Index: 629

Text #7

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7857. Min Act: -0.0747

Data Index: 6893976 (Open Web Text)

Max Activating Token Index: 613

Text #8

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7890. Min Act: -0.0747

Data Index: 6893971 (Open Web Text)

Max Activating Token Index: 60

Text #9

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7767. Min Act: -0.0882

Data Index: 23 (Open Web Text)

Max Activating Token Index: 761

Text #10

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7877. Min Act: -0.0747

Data Index: 5474850 (Open Web Text)

Max Activating Token Index: 962

Text #11

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7793. Min Act: -0.1637

Data Index: 6893970 (Open Web Text)

Max Activating Token Index: 1022

Text #12

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7823. Min Act: -0.0882

Data Index: 549097 (Open Web Text)

Max Activating Token Index: 1015

Text #13

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7641. Min Act: -0.1589

Data Index: 6404273 (Open Web Text)

Max Activating Token Index: 906

Text #14

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7662. Min Act: -0.1538

Data Index: 8354494 (Open Web Text)

Max Activating Token Index: 333

Text #15

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7631. Min Act: -0.1625

Data Index: 373617 (Open Web Text)

Max Activating Token Index: 517

Text #16

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7659. Min Act: -0.0810

Data Index: 6652786 (Open Web Text)

Max Activating Token Index: 320

Text #17

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7683. Min Act: -0.0747

Data Index: 3100722 (Open Web Text)

Max Activating Token Index: 537

Text #18

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7681. Min Act: -0.1193

Data Index: 6023373 (Open Web Text)

Max Activating Token Index: 566

Text #19

Max Range: 1.9098. Min Range: -1.9098

Max Act: 1.7692. Min Act: -0.0747

Data Index: 3646258 (Open Web Text)

Max Activating Token Index: 504