Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2116 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1466. Min Range: -3.1466

Max Act: 3.1466. Min Act: -0.1700

Data Index: 5854874 (Open Web Text)

Max Activating Token Index: 71

Text #1

Max Range: 3.1466. Min Range: -3.1466

Max Act: 3.1407. Min Act: -0.1700

Data Index: 3964137 (Open Web Text)

Max Activating Token Index: 614

Text #2

Max Range: 3.1466. Min Range: -3.1466

Max Act: 3.0472. Min Act: -0.1700

Data Index: 1633593 (Open Web Text)

Max Activating Token Index: 444

Text #3

Max Range: 3.1466. Min Range: -3.1466

Max Act: 3.0336. Min Act: -0.1700

Data Index: 7363221 (Open Web Text)

Max Activating Token Index: 928

Text #4

Max Range: 3.1466. Min Range: -3.1466

Max Act: 3.0445. Min Act: -0.1700

Data Index: 2316925 (Open Web Text)

Max Activating Token Index: 608

Text #5

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9882. Min Act: -0.1700

Data Index: 7469658 (Open Web Text)

Max Activating Token Index: 656

Text #6

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9863. Min Act: -0.1700

Data Index: 2005476 (Open Web Text)

Max Activating Token Index: 261

Text #7

Max Range: 3.1466. Min Range: -3.1466

Max Act: 3.0035. Min Act: -0.1700

Data Index: 3446785 (Open Web Text)

Max Activating Token Index: 320

Text #8

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9832. Min Act: -0.1700

Data Index: 1313562 (Open Web Text)

Max Activating Token Index: 200

Text #9

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9946. Min Act: -0.1700

Data Index: 2792167 (Open Web Text)

Max Activating Token Index: 108

Text #10

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9570. Min Act: -0.1700

Data Index: 7467152 (Open Web Text)

Max Activating Token Index: 783

Text #11

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9725. Min Act: -0.1700

Data Index: 3658356 (Open Web Text)

Max Activating Token Index: 930

Text #12

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9629. Min Act: -0.1700

Data Index: 2566611 (Open Web Text)

Max Activating Token Index: 824

Text #13

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9740. Min Act: -0.1700

Data Index: 8112495 (Open Web Text)

Max Activating Token Index: 360

Text #14

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9783. Min Act: -0.1700

Data Index: 4964464 (Open Web Text)

Max Activating Token Index: 492

Text #15

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9555. Min Act: -0.1700

Data Index: 7173988 (Open Web Text)

Max Activating Token Index: 918

Text #16

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9849. Min Act: -0.1700

Data Index: 6468888 (Open Web Text)

Max Activating Token Index: 417

Text #17

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9798. Min Act: -0.1700

Data Index: 8423634 (Open Web Text)

Max Activating Token Index: 180

Text #18

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.9512. Min Act: -0.1700

Data Index: 3029127 (Open Web Text)

Max Activating Token Index: 939

Text #19

Max Range: 3.1466. Min Range: -3.1466

Max Act: 2.8395. Min Act: -0.1700

Data Index: 1795984 (Open Web Text)

Max Activating Token Index: 785