Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1343 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.7554. Min Act: -0.1700

Data Index: 7022991 (Open Web Text)

Max Activating Token Index: 842

Text #1

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.7499. Min Act: -0.1700

Data Index: 8386107 (Open Web Text)

Max Activating Token Index: 630

Text #2

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.7338. Min Act: -0.1700

Data Index: 6167959 (Open Web Text)

Max Activating Token Index: 836

Text #3

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.7051. Min Act: -0.1700

Data Index: 944420 (Open Web Text)

Max Activating Token Index: 964

Text #4

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.6961. Min Act: -0.1700

Data Index: 6374265 (Open Web Text)

Max Activating Token Index: 473

Text #5

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.7106. Min Act: -0.1700

Data Index: 8241184 (Open Web Text)

Max Activating Token Index: 569

Text #6

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.6467. Min Act: -0.1700

Data Index: 7461230 (Open Web Text)

Max Activating Token Index: 999

Text #7

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.6592. Min Act: -0.1700

Data Index: 905153 (Open Web Text)

Max Activating Token Index: 649

Text #8

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.6556. Min Act: -0.1700

Data Index: 7100652 (Open Web Text)

Max Activating Token Index: 704

Text #9

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.6318. Min Act: -0.1700

Data Index: 1887768 (Open Web Text)

Max Activating Token Index: 520

Text #10

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.6325. Min Act: -0.1700

Data Index: 2562769 (Open Web Text)

Max Activating Token Index: 458

Text #11

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.6652. Min Act: -0.1700

Data Index: 5261343 (Open Web Text)

Max Activating Token Index: 798

Text #12

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.6367. Min Act: -0.1700

Data Index: 6893928 (Open Web Text)

Max Activating Token Index: 414

Text #13

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.5966. Min Act: -0.1700

Data Index: 2389598 (Open Web Text)

Max Activating Token Index: 400

Text #14

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.5623. Min Act: -0.1700

Data Index: 6778375 (Open Web Text)

Max Activating Token Index: 621

Text #15

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.5784. Min Act: -0.1700

Data Index: 5856798 (Open Web Text)

Max Activating Token Index: 367

Text #16

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.5558. Min Act: -0.1700

Data Index: 2892393 (Open Web Text)

Max Activating Token Index: 794

Text #17

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.5570. Min Act: -0.1700

Data Index: 958125 (Open Web Text)

Max Activating Token Index: 709

Text #18

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.6045. Min Act: -0.1700

Data Index: 8229320 (Open Web Text)

Max Activating Token Index: 947

Text #19

Max Range: 4.7554. Min Range: -4.7554

Max Act: 4.5886. Min Act: -0.1700

Data Index: 868419 (Open Web Text)

Max Activating Token Index: 637