Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1587 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.9876. Min Act: -0.1700

Data Index: 6732328 (Open Web Text)

Max Activating Token Index: 106

Text #1

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.9684. Min Act: -0.1700

Data Index: 7879334 (Open Web Text)

Max Activating Token Index: 432

Text #2

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8596. Min Act: -0.1700

Data Index: 394393 (Open Web Text)

Max Activating Token Index: 336

Text #3

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8632. Min Act: -0.1700

Data Index: 7530895 (Open Web Text)

Max Activating Token Index: 143

Text #4

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8425. Min Act: -0.1700

Data Index: 6905487 (Open Web Text)

Max Activating Token Index: 402

Text #5

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.7916. Min Act: -0.1700

Data Index: 8061670 (Open Web Text)

Max Activating Token Index: 770

Text #6

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.7871. Min Act: -0.1700

Data Index: 3374834 (Open Web Text)

Max Activating Token Index: 967

Text #7

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.7720. Min Act: -0.1700

Data Index: 449753 (Open Web Text)

Max Activating Token Index: 894

Text #8

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.7497. Min Act: -0.1700

Data Index: 3946382 (Open Web Text)

Max Activating Token Index: 654

Text #9

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.7375. Min Act: -0.1700

Data Index: 6865025 (Open Web Text)

Max Activating Token Index: 645

Text #10

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.7334. Min Act: -0.1700

Data Index: 234007 (Open Web Text)

Max Activating Token Index: 447

Text #11

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.6730. Min Act: -0.1700

Data Index: 7541096 (Open Web Text)

Max Activating Token Index: 839

Text #12

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.6694. Min Act: -0.1700

Data Index: 7164045 (Open Web Text)

Max Activating Token Index: 524

Text #13

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.7082. Min Act: -0.1700

Data Index: 4239747 (Open Web Text)

Max Activating Token Index: 617

Text #14

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.6815. Min Act: -0.1700

Data Index: 1348727 (Open Web Text)

Max Activating Token Index: 194

Text #15

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.6799. Min Act: -0.1700

Data Index: 5504334 (Open Web Text)

Max Activating Token Index: 262

Text #16

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.7020. Min Act: -0.1700

Data Index: 7298657 (Open Web Text)

Max Activating Token Index: 682

Text #17

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.6740. Min Act: -0.1700

Data Index: 1508657 (Open Web Text)

Max Activating Token Index: 280

Text #18

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.6584. Min Act: -0.1700

Data Index: 6249744 (Open Web Text)

Max Activating Token Index: 60

Text #19

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.6780. Min Act: -0.1700

Data Index: 2844443 (Open Web Text)

Max Activating Token Index: 837