Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 274 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2988. Min Range: -4.2988

Max Act: 4.2988. Min Act: -0.1700

Data Index: 2461349 (Open Web Text)

Max Activating Token Index: 816

Text #1

Max Range: 4.2988. Min Range: -4.2988

Max Act: 4.0305. Min Act: -0.1700

Data Index: 6928136 (Open Web Text)

Max Activating Token Index: 412

Text #2

Max Range: 4.2988. Min Range: -4.2988

Max Act: 4.0018. Min Act: -0.1700

Data Index: 6137668 (Open Web Text)

Max Activating Token Index: 893

Text #3

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.9823. Min Act: -0.1700

Data Index: 1176134 (Open Web Text)

Max Activating Token Index: 564

Text #4

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.9136. Min Act: -0.1700

Data Index: 7000491 (Open Web Text)

Max Activating Token Index: 345

Text #5

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.9538. Min Act: -0.1700

Data Index: 3438221 (Open Web Text)

Max Activating Token Index: 841

Text #6

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.9370. Min Act: -0.1700

Data Index: 1758033 (Open Web Text)

Max Activating Token Index: 393

Text #7

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.9131. Min Act: -0.1700

Data Index: 6645413 (Open Web Text)

Max Activating Token Index: 751

Text #8

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.8954. Min Act: -0.1700

Data Index: 7139374 (Open Web Text)

Max Activating Token Index: 584

Text #9

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.8796. Min Act: -0.1700

Data Index: 6645412 (Open Web Text)

Max Activating Token Index: 822

Text #10

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.8640. Min Act: -0.1700

Data Index: 1108602 (Open Web Text)

Max Activating Token Index: 435

Text #11

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.8470. Min Act: -0.1700

Data Index: 1341109 (Open Web Text)

Max Activating Token Index: 914

Text #12

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.8701. Min Act: -0.1700

Data Index: 1562694 (Open Web Text)

Max Activating Token Index: 659

Text #13

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.8494. Min Act: -0.1700

Data Index: 632266 (Open Web Text)

Max Activating Token Index: 495

Text #14

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.8268. Min Act: -0.1700

Data Index: 5780977 (Open Web Text)

Max Activating Token Index: 271

Text #15

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.7999. Min Act: -0.1700

Data Index: 2121493 (Open Web Text)

Max Activating Token Index: 948

Text #16

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.7828. Min Act: -0.1700

Data Index: 1797173 (Open Web Text)

Max Activating Token Index: 637

Text #17

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.7558. Min Act: -0.1700

Data Index: 7617540 (Open Web Text)

Max Activating Token Index: 454

Text #18

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.7693. Min Act: -0.1700

Data Index: 6670868 (Open Web Text)

Max Activating Token Index: 894

Text #19

Max Range: 4.2988. Min Range: -4.2988

Max Act: 3.7621. Min Act: -0.1700

Data Index: 2379378 (Open Web Text)

Max Activating Token Index: 539