Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1035 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.8901. Min Act: -0.1700

Data Index: 3529280 (Open Web Text)

Max Activating Token Index: 357

Text #1

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.8736. Min Act: -0.1700

Data Index: 510021 (Open Web Text)

Max Activating Token Index: 413

Text #2

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.8327. Min Act: -0.1700

Data Index: 8123534 (Open Web Text)

Max Activating Token Index: 414

Text #3

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.8312. Min Act: -0.1700

Data Index: 2642903 (Open Web Text)

Max Activating Token Index: 383

Text #4

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.8111. Min Act: -0.1700

Data Index: 1917520 (Open Web Text)

Max Activating Token Index: 32

Text #5

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.8073. Min Act: -0.1700

Data Index: 7274967 (Open Web Text)

Max Activating Token Index: 250

Text #6

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.8084. Min Act: -0.1698

Data Index: 8655787 (Open Web Text)

Max Activating Token Index: 668

Text #7

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.8104. Min Act: -0.1700

Data Index: 5606026 (Open Web Text)

Max Activating Token Index: 242

Text #8

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.7647. Min Act: -0.1700

Data Index: 3562450 (Open Web Text)

Max Activating Token Index: 526

Text #9

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.7717. Min Act: -0.1700

Data Index: 7665245 (Open Web Text)

Max Activating Token Index: 838

Text #10

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.7805. Min Act: -0.1700

Data Index: 3615976 (Open Web Text)

Max Activating Token Index: 69

Text #11

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.7749. Min Act: -0.1700

Data Index: 7222627 (Open Web Text)

Max Activating Token Index: 787

Text #12

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.7765. Min Act: -0.1700

Data Index: 7915349 (Open Web Text)

Max Activating Token Index: 34

Text #13

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.7736. Min Act: -0.1700

Data Index: 4807480 (Open Web Text)

Max Activating Token Index: 348

Text #14

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.7831. Min Act: -0.1700

Data Index: 3659818 (Open Web Text)

Max Activating Token Index: 368

Text #15

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.7862. Min Act: -0.1700

Data Index: 6844996 (Open Web Text)

Max Activating Token Index: 873

Text #16

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.7507. Min Act: -0.1700

Data Index: 7722903 (Open Web Text)

Max Activating Token Index: 452

Text #17

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.7381. Min Act: -0.1700

Data Index: 7013774 (Open Web Text)

Max Activating Token Index: 797

Text #18

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.7457. Min Act: -0.1699

Data Index: 933423 (Open Web Text)

Max Activating Token Index: 305

Text #19

Max Range: 1.8901. Min Range: -1.8901

Max Act: 1.7432. Min Act: -0.1700

Data Index: 7243626 (Open Web Text)

Max Activating Token Index: 34