Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2138 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0515. Min Range: -3.0515

Max Act: 3.0515. Min Act: -0.1700

Data Index: 5719151 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 3.0515. Min Range: -3.0515

Max Act: 3.0420. Min Act: -0.1700

Data Index: 7882445 (Open Web Text)

Max Activating Token Index: 580

Text #2

Max Range: 3.0515. Min Range: -3.0515

Max Act: 3.0202. Min Act: -0.1700

Data Index: 8146244 (Open Web Text)

Max Activating Token Index: 644

Text #3

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.9521. Min Act: -0.1700

Data Index: 5687168 (Open Web Text)

Max Activating Token Index: 910

Text #4

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.9511. Min Act: -0.1700

Data Index: 7724902 (Open Web Text)

Max Activating Token Index: 511

Text #5

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.9459. Min Act: -0.1700

Data Index: 4854812 (Open Web Text)

Max Activating Token Index: 325

Text #6

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.9192. Min Act: -0.1700

Data Index: 1826920 (Open Web Text)

Max Activating Token Index: 355

Text #7

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.9108. Min Act: -0.1700

Data Index: 6750867 (Open Web Text)

Max Activating Token Index: 442

Text #8

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.8941. Min Act: -0.1700

Data Index: 3243545 (Open Web Text)

Max Activating Token Index: 466

Text #9

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.8863. Min Act: -0.1700

Data Index: 2743250 (Open Web Text)

Max Activating Token Index: 232

Text #10

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.8761. Min Act: -0.1700

Data Index: 5112507 (Open Web Text)

Max Activating Token Index: 672

Text #11

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.8766. Min Act: -0.1700

Data Index: 2394613 (Open Web Text)

Max Activating Token Index: 371

Text #12

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.8755. Min Act: -0.1700

Data Index: 2649791 (Open Web Text)

Max Activating Token Index: 879

Text #13

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.8456. Min Act: -0.1700

Data Index: 3671685 (Open Web Text)

Max Activating Token Index: 841

Text #14

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.8324. Min Act: -0.1700

Data Index: 6352629 (Open Web Text)

Max Activating Token Index: 963

Text #15

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.8469. Min Act: -0.1700

Data Index: 6202091 (Open Web Text)

Max Activating Token Index: 66

Text #16

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.8254. Min Act: -0.1700

Data Index: 5977282 (Open Web Text)

Max Activating Token Index: 446

Text #17

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.7812. Min Act: -0.1700

Data Index: 4250911 (Open Web Text)

Max Activating Token Index: 694

Text #18

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.8077. Min Act: -0.1700

Data Index: 1621815 (Open Web Text)

Max Activating Token Index: 993

Text #19

Max Range: 3.0515. Min Range: -3.0515

Max Act: 2.7932. Min Act: -0.1700

Data Index: 7452443 (Open Web Text)

Max Activating Token Index: 925