Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1923 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.8071. Min Act: -0.1700

Data Index: 3227835 (Open Web Text)

Max Activating Token Index: 354

Text #1

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.8216. Min Act: -0.1700

Data Index: 6936883 (Open Web Text)

Max Activating Token Index: 289

Text #2

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.7751. Min Act: -0.1700

Data Index: 6460354 (Open Web Text)

Max Activating Token Index: 375

Text #3

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.7869. Min Act: -0.1700

Data Index: 190417 (Open Web Text)

Max Activating Token Index: 145

Text #4

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.7798. Min Act: -0.1700

Data Index: 5028828 (Open Web Text)

Max Activating Token Index: 990

Text #5

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.7283. Min Act: -0.1700

Data Index: 3227814 (Open Web Text)

Max Activating Token Index: 914

Text #6

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.6802. Min Act: -0.1700

Data Index: 6936867 (Open Web Text)

Max Activating Token Index: 603

Text #7

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.6774. Min Act: -0.1700

Data Index: 190426 (Open Web Text)

Max Activating Token Index: 100

Text #8

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.6462. Min Act: -0.1700

Data Index: 7588427 (Open Web Text)

Max Activating Token Index: 713

Text #9

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.5722. Min Act: -0.1700

Data Index: 2339479 (Open Web Text)

Max Activating Token Index: 766

Text #10

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.5290. Min Act: -0.1700

Data Index: 3227811 (Open Web Text)

Max Activating Token Index: 594

Text #11

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.5060. Min Act: -0.1700

Data Index: 2885946 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4656. Min Act: -0.1700

Data Index: 2339493 (Open Web Text)

Max Activating Token Index: 487

Text #13

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4819. Min Act: -0.1700

Data Index: 8563798 (Open Web Text)

Max Activating Token Index: 21

Text #14

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4422. Min Act: -0.1700

Data Index: 8289558 (Open Web Text)

Max Activating Token Index: 872

Text #15

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4482. Min Act: -0.1700

Data Index: 932109 (Open Web Text)

Max Activating Token Index: 718

Text #16

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3878. Min Act: -0.1700

Data Index: 1456605 (Open Web Text)

Max Activating Token Index: 619

Text #17

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4099. Min Act: -0.1700

Data Index: 5241314 (Open Web Text)

Max Activating Token Index: 209

Text #18

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3529. Min Act: -0.1700

Data Index: 1877599 (Open Web Text)

Max Activating Token Index: 317

Text #19

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3480. Min Act: -0.1700

Data Index: 2215012 (Open Web Text)

Max Activating Token Index: 67