Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 768 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0509. Min Range: -3.0509

Max Act: 3.0471. Min Act: -0.1700

Data Index: 3651349 (Open Web Text)

Max Activating Token Index: 768

Text #1

Max Range: 3.0509. Min Range: -3.0509

Max Act: 3.0509. Min Act: -0.1700

Data Index: 1488489 (Open Web Text)

Max Activating Token Index: 616

Text #2

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.9765. Min Act: -0.1700

Data Index: 6543938 (Open Web Text)

Max Activating Token Index: 1006

Text #3

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.9381. Min Act: -0.1700

Data Index: 3675079 (Open Web Text)

Max Activating Token Index: 955

Text #4

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.9274. Min Act: -0.1700

Data Index: 5935597 (Open Web Text)

Max Activating Token Index: 569

Text #5

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.9344. Min Act: -0.1700

Data Index: 7291831 (Open Web Text)

Max Activating Token Index: 378

Text #6

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.9413. Min Act: -0.1700

Data Index: 8445433 (Open Web Text)

Max Activating Token Index: 93

Text #7

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.9235. Min Act: -0.1700

Data Index: 7908141 (Open Web Text)

Max Activating Token Index: 748

Text #8

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.9108. Min Act: -0.1700

Data Index: 5171196 (Open Web Text)

Max Activating Token Index: 632

Text #9

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.9287. Min Act: -0.1700

Data Index: 5330784 (Open Web Text)

Max Activating Token Index: 456

Text #10

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.9004. Min Act: -0.1700

Data Index: 4954123 (Open Web Text)

Max Activating Token Index: 707

Text #11

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.8923. Min Act: -0.1700

Data Index: 620463 (Open Web Text)

Max Activating Token Index: 627

Text #12

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.8995. Min Act: -0.1700

Data Index: 4149338 (Open Web Text)

Max Activating Token Index: 698

Text #13

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.8873. Min Act: -0.1700

Data Index: 5389470 (Open Web Text)

Max Activating Token Index: 553

Text #14

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.8862. Min Act: -0.1700

Data Index: 3679463 (Open Web Text)

Max Activating Token Index: 790

Text #15

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.8786. Min Act: -0.1700

Data Index: 4618905 (Open Web Text)

Max Activating Token Index: 828

Text #16

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.8971. Min Act: -0.1700

Data Index: 1816621 (Open Web Text)

Max Activating Token Index: 168

Text #17

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.8965. Min Act: -0.1700

Data Index: 4746474 (Open Web Text)

Max Activating Token Index: 836

Text #18

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.8834. Min Act: -0.1700

Data Index: 5110347 (Open Web Text)

Max Activating Token Index: 679

Text #19

Max Range: 3.0509. Min Range: -3.0509

Max Act: 2.8568. Min Act: -0.1700

Data Index: 6373259 (Open Web Text)

Max Activating Token Index: 719