Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 432 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4575. Min Range: -4.4575

Max Act: 4.4575. Min Act: -0.1700

Data Index: 5102709 (Open Web Text)

Max Activating Token Index: 623

Text #1

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.9500. Min Act: -0.1700

Data Index: 3980726 (Open Web Text)

Max Activating Token Index: 786

Text #2

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.7537. Min Act: -0.1700

Data Index: 6014356 (Open Web Text)

Max Activating Token Index: 452

Text #3

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.7659. Min Act: -0.1700

Data Index: 6257714 (Open Web Text)

Max Activating Token Index: 382

Text #4

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.7942. Min Act: -0.1700

Data Index: 6603694 (Open Web Text)

Max Activating Token Index: 934

Text #5

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.7830. Min Act: -0.1700

Data Index: 1208529 (Open Web Text)

Max Activating Token Index: 993

Text #6

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.7086. Min Act: -0.1700

Data Index: 5452696 (Open Web Text)

Max Activating Token Index: 509

Text #7

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.7279. Min Act: -0.1700

Data Index: 207199 (Open Web Text)

Max Activating Token Index: 529

Text #8

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.6407. Min Act: -0.1700

Data Index: 5178672 (Open Web Text)

Max Activating Token Index: 481

Text #9

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.6851. Min Act: -0.1700

Data Index: 6704086 (Open Web Text)

Max Activating Token Index: 877

Text #10

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.6327. Min Act: -0.1699

Data Index: 1446516 (Open Web Text)

Max Activating Token Index: 159

Text #11

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.6839. Min Act: -0.1700

Data Index: 2382290 (Open Web Text)

Max Activating Token Index: 1008

Text #12

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.6420. Min Act: -0.1700

Data Index: 3488834 (Open Web Text)

Max Activating Token Index: 708

Text #13

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.6557. Min Act: -0.1700

Data Index: 1714054 (Open Web Text)

Max Activating Token Index: 674

Text #14

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.6626. Min Act: -0.1700

Data Index: 6450524 (Open Web Text)

Max Activating Token Index: 322

Text #15

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.6273. Min Act: -0.1700

Data Index: 8667789 (Open Web Text)

Max Activating Token Index: 979

Text #16

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.6569. Min Act: -0.1700

Data Index: 858748 (Open Web Text)

Max Activating Token Index: 646

Text #17

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.6638. Min Act: -0.1700

Data Index: 5316425 (Open Web Text)

Max Activating Token Index: 120

Text #18

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.5779. Min Act: -0.1700

Data Index: 7319919 (Open Web Text)

Max Activating Token Index: 994

Text #19

Max Range: 4.4575. Min Range: -4.4575

Max Act: 3.6397. Min Act: -0.1700

Data Index: 8097766 (Open Web Text)

Max Activating Token Index: 162