Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2451 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.8072. Min Act: -0.1102

Data Index: 1879026 (Open Web Text)

Max Activating Token Index: 7

Text #1

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.7811. Min Act: -0.1212

Data Index: 4543222 (Open Web Text)

Max Activating Token Index: 982

Text #2

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.7636. Min Act: -0.1145

Data Index: 4644936 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.7525. Min Act: -0.1133

Data Index: 7444311 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.7187. Min Act: -0.1102

Data Index: 46011 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.7142. Min Act: -0.1034

Data Index: 5449278 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.7082. Min Act: -0.0847

Data Index: 7406941 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.7093. Min Act: -0.1273

Data Index: 3414652 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.7043. Min Act: -0.1222

Data Index: 2671862 (Open Web Text)

Max Activating Token Index: 7

Text #9

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.6960. Min Act: -0.0805

Data Index: 5928114 (Open Web Text)

Max Activating Token Index: 7

Text #10

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.6927. Min Act: -0.1094

Data Index: 1793003 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.6664. Min Act: -0.0864

Data Index: 4865074 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.6664. Min Act: -0.1579

Data Index: 5304011 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.6705. Min Act: -0.1282

Data Index: 4205515 (Open Web Text)

Max Activating Token Index: 6

Text #14

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.6664. Min Act: -0.1503

Data Index: 4543223 (Open Web Text)

Max Activating Token Index: 7

Text #15

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.6733. Min Act: -0.1309

Data Index: 1356024 (Open Web Text)

Max Activating Token Index: 10

Text #16

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.6679. Min Act: -0.1172

Data Index: 4430095 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.6681. Min Act: -0.1340

Data Index: 5092653 (Open Web Text)

Max Activating Token Index: 8

Text #18

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.6664. Min Act: 0.0378

Data Index: 1609535 (Open Web Text)

Max Activating Token Index: 7

Text #19

Max Range: 0.8072. Min Range: -0.8072

Max Act: 0.6664. Min Act: -0.1217

Data Index: 5832301 (Open Web Text)

Max Activating Token Index: 7