Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1951 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.7521. Min Act: -0.1700

Data Index: 2522621 (Open Web Text)

Max Activating Token Index: 153

Text #1

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.6778. Min Act: -0.1700

Data Index: 7939342 (Open Web Text)

Max Activating Token Index: 78

Text #2

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5996. Min Act: -0.1700

Data Index: 234463 (Open Web Text)

Max Activating Token Index: 574

Text #3

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.6112. Min Act: -0.1700

Data Index: 534187 (Open Web Text)

Max Activating Token Index: 31

Text #4

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5760. Min Act: -0.1700

Data Index: 5139342 (Open Web Text)

Max Activating Token Index: 764

Text #5

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5811. Min Act: -0.1700

Data Index: 3026811 (Open Web Text)

Max Activating Token Index: 1018

Text #6

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5634. Min Act: -0.1700

Data Index: 7103882 (Open Web Text)

Max Activating Token Index: 907

Text #7

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5644. Min Act: -0.1700

Data Index: 893633 (Open Web Text)

Max Activating Token Index: 304

Text #8

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5683. Min Act: -0.1700

Data Index: 4497639 (Open Web Text)

Max Activating Token Index: 748

Text #9

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5812. Min Act: -0.1700

Data Index: 6588559 (Open Web Text)

Max Activating Token Index: 774

Text #10

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5445. Min Act: -0.1700

Data Index: 2626078 (Open Web Text)

Max Activating Token Index: 445

Text #11

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5495. Min Act: -0.1700

Data Index: 503315 (Open Web Text)

Max Activating Token Index: 549

Text #12

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5668. Min Act: -0.1700

Data Index: 6465070 (Open Web Text)

Max Activating Token Index: 620

Text #13

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5470. Min Act: -0.1700

Data Index: 6120752 (Open Web Text)

Max Activating Token Index: 408

Text #14

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5478. Min Act: -0.1700

Data Index: 502655 (Open Web Text)

Max Activating Token Index: 802

Text #15

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5566. Min Act: -0.1700

Data Index: 8147873 (Open Web Text)

Max Activating Token Index: 619

Text #16

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5582. Min Act: -0.1700

Data Index: 3717190 (Open Web Text)

Max Activating Token Index: 470

Text #17

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5099. Min Act: -0.1700

Data Index: 6674025 (Open Web Text)

Max Activating Token Index: 922

Text #18

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5060. Min Act: -0.1700

Data Index: 6789371 (Open Web Text)

Max Activating Token Index: 540

Text #19

Max Range: 3.7521. Min Range: -3.7521

Max Act: 3.5477. Min Act: -0.1700

Data Index: 2545720 (Open Web Text)

Max Activating Token Index: 62