Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1434 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.3509. Min Act: -0.1700

Data Index: 2845867 (Open Web Text)

Max Activating Token Index: 998

Text #1

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.3126. Min Act: -0.1700

Data Index: 379686 (Open Web Text)

Max Activating Token Index: 512

Text #2

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2798. Min Act: -0.1700

Data Index: 8120113 (Open Web Text)

Max Activating Token Index: 822

Text #3

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2944. Min Act: -0.1700

Data Index: 5979635 (Open Web Text)

Max Activating Token Index: 756

Text #4

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.3045. Min Act: -0.1700

Data Index: 5558850 (Open Web Text)

Max Activating Token Index: 577

Text #5

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2884. Min Act: -0.1700

Data Index: 1367030 (Open Web Text)

Max Activating Token Index: 947

Text #6

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2825. Min Act: -0.1700

Data Index: 4438463 (Open Web Text)

Max Activating Token Index: 580

Text #7

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2644. Min Act: -0.1700

Data Index: 6157662 (Open Web Text)

Max Activating Token Index: 422

Text #8

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2481. Min Act: -0.1700

Data Index: 2067918 (Open Web Text)

Max Activating Token Index: 341

Text #9

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2533. Min Act: -0.1700

Data Index: 6535797 (Open Web Text)

Max Activating Token Index: 684

Text #10

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2687. Min Act: -0.1700

Data Index: 3712724 (Open Web Text)

Max Activating Token Index: 697

Text #11

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2478. Min Act: -0.1700

Data Index: 3917084 (Open Web Text)

Max Activating Token Index: 802

Text #12

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2499. Min Act: -0.1700

Data Index: 4684405 (Open Web Text)

Max Activating Token Index: 1006

Text #13

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2362. Min Act: -0.1700

Data Index: 7091116 (Open Web Text)

Max Activating Token Index: 857

Text #14

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2476. Min Act: -0.1700

Data Index: 1763172 (Open Web Text)

Max Activating Token Index: 313

Text #15

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2521. Min Act: -0.1700

Data Index: 8299489 (Open Web Text)

Max Activating Token Index: 638

Text #16

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2329. Min Act: -0.1700

Data Index: 6569067 (Open Web Text)

Max Activating Token Index: 371

Text #17

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2296. Min Act: -0.1700

Data Index: 7728558 (Open Web Text)

Max Activating Token Index: 680

Text #18

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2398. Min Act: -0.1700

Data Index: 4691146 (Open Web Text)

Max Activating Token Index: 1002

Text #19

Max Range: 3.3509. Min Range: -3.3509

Max Act: 3.2441. Min Act: -0.1700

Data Index: 7856473 (Open Web Text)

Max Activating Token Index: 760