Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1427 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.8013. Min Act: -0.1700

Data Index: 5013157 (Open Web Text)

Max Activating Token Index: 549

Text #1

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7690. Min Act: -0.1700

Data Index: 3224036 (Open Web Text)

Max Activating Token Index: 630

Text #2

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7600. Min Act: -0.1700

Data Index: 2775413 (Open Web Text)

Max Activating Token Index: 596

Text #3

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7548. Min Act: -0.1700

Data Index: 695903 (Open Web Text)

Max Activating Token Index: 632

Text #4

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7522. Min Act: -0.1700

Data Index: 5673722 (Open Web Text)

Max Activating Token Index: 613

Text #5

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7544. Min Act: -0.1700

Data Index: 42923 (Open Web Text)

Max Activating Token Index: 564

Text #6

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7593. Min Act: -0.1700

Data Index: 3642283 (Open Web Text)

Max Activating Token Index: 627

Text #7

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7532. Min Act: -0.1700

Data Index: 3368683 (Open Web Text)

Max Activating Token Index: 612

Text #8

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7407. Min Act: -0.1700

Data Index: 5649653 (Open Web Text)

Max Activating Token Index: 641

Text #9

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7374. Min Act: -0.1700

Data Index: 3844997 (Open Web Text)

Max Activating Token Index: 615

Text #10

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7338. Min Act: -0.1700

Data Index: 8674812 (Open Web Text)

Max Activating Token Index: 625

Text #11

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7535. Min Act: -0.1700

Data Index: 185410 (Open Web Text)

Max Activating Token Index: 604

Text #12

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7354. Min Act: -0.1700

Data Index: 2486281 (Open Web Text)

Max Activating Token Index: 596

Text #13

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7475. Min Act: -0.1700

Data Index: 5773399 (Open Web Text)

Max Activating Token Index: 595

Text #14

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7371. Min Act: -0.1700

Data Index: 439210 (Open Web Text)

Max Activating Token Index: 628

Text #15

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7369. Min Act: -0.1700

Data Index: 3563889 (Open Web Text)

Max Activating Token Index: 600

Text #16

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7471. Min Act: -0.1700

Data Index: 3196708 (Open Web Text)

Max Activating Token Index: 611

Text #17

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7262. Min Act: -0.1700

Data Index: 5416953 (Open Web Text)

Max Activating Token Index: 595

Text #18

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7390. Min Act: -0.1700

Data Index: 2788074 (Open Web Text)

Max Activating Token Index: 616

Text #19

Max Range: 0.8013. Min Range: -0.8013

Max Act: 0.7194. Min Act: -0.1700

Data Index: 7906935 (Open Web Text)

Max Activating Token Index: 613