Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2062 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.6636. Min Act: -0.1700

Data Index: 4271989 (Open Web Text)

Max Activating Token Index: 820

Text #1

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.6574. Min Act: -0.1698

Data Index: 6011602 (Open Web Text)

Max Activating Token Index: 754

Text #2

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.6533. Min Act: -0.1700

Data Index: 5215174 (Open Web Text)

Max Activating Token Index: 683

Text #3

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.6259. Min Act: -0.1700

Data Index: 4499832 (Open Web Text)

Max Activating Token Index: 524

Text #4

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5945. Min Act: -0.1700

Data Index: 1899807 (Open Web Text)

Max Activating Token Index: 797

Text #5

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5872. Min Act: -0.1700

Data Index: 4917552 (Open Web Text)

Max Activating Token Index: 8

Text #6

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5665. Min Act: -0.1700

Data Index: 4277755 (Open Web Text)

Max Activating Token Index: 13

Text #7

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5627. Min Act: -0.1700

Data Index: 2060784 (Open Web Text)

Max Activating Token Index: 619

Text #8

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5557. Min Act: -0.1700

Data Index: 5869225 (Open Web Text)

Max Activating Token Index: 747

Text #9

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5627. Min Act: -0.1700

Data Index: 2658932 (Open Web Text)

Max Activating Token Index: 1016

Text #10

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5602. Min Act: -0.1698

Data Index: 4483758 (Open Web Text)

Max Activating Token Index: 870

Text #11

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5413. Min Act: -0.1700

Data Index: 215323 (Open Web Text)

Max Activating Token Index: 741

Text #12

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5382. Min Act: -0.1700

Data Index: 6949333 (Open Web Text)

Max Activating Token Index: 539

Text #13

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5500. Min Act: -0.1700

Data Index: 1311875 (Open Web Text)

Max Activating Token Index: 757

Text #14

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5319. Min Act: -0.1700

Data Index: 7157027 (Open Web Text)

Max Activating Token Index: 61

Text #15

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5225. Min Act: -0.1700

Data Index: 1632228 (Open Web Text)

Max Activating Token Index: 738

Text #16

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5334. Min Act: -0.1700

Data Index: 2778567 (Open Web Text)

Max Activating Token Index: 6

Text #17

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5260. Min Act: -0.1700

Data Index: 5734157 (Open Web Text)

Max Activating Token Index: 10

Text #18

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5288. Min Act: -0.1700

Data Index: 4940631 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 1.6636. Min Range: -1.6636

Max Act: 1.5352. Min Act: -0.1700

Data Index: 600474 (Open Web Text)

Max Activating Token Index: 13