Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2679 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.0352. Min Range: -2.0352

Max Act: 2.0336. Min Act: -0.1700

Data Index: 158270 (Open Web Text)

Max Activating Token Index: 718

Text #1

Max Range: 2.0352. Min Range: -2.0352

Max Act: 2.0352. Min Act: -0.1700

Data Index: 4802775 (Open Web Text)

Max Activating Token Index: 33

Text #2

Max Range: 2.0352. Min Range: -2.0352

Max Act: 2.0086. Min Act: -0.1700

Data Index: 392327 (Open Web Text)

Max Activating Token Index: 339

Text #3

Max Range: 2.0352. Min Range: -2.0352

Max Act: 2.0021. Min Act: -0.1700

Data Index: 6122674 (Open Web Text)

Max Activating Token Index: 169

Text #4

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.9648. Min Act: -0.1700

Data Index: 4135639 (Open Web Text)

Max Activating Token Index: 99

Text #5

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.9365. Min Act: -0.1700

Data Index: 3907247 (Open Web Text)

Max Activating Token Index: 16

Text #6

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.9198. Min Act: -0.1700

Data Index: 1955961 (Open Web Text)

Max Activating Token Index: 359

Text #7

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.9246. Min Act: -0.1700

Data Index: 5869093 (Open Web Text)

Max Activating Token Index: 929

Text #8

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.9190. Min Act: -0.1700

Data Index: 1708052 (Open Web Text)

Max Activating Token Index: 30

Text #9

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.9141. Min Act: -0.1700

Data Index: 5613120 (Open Web Text)

Max Activating Token Index: 33

Text #10

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.9086. Min Act: -0.1700

Data Index: 4883014 (Open Web Text)

Max Activating Token Index: 24

Text #11

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.9055. Min Act: -0.1700

Data Index: 5609937 (Open Web Text)

Max Activating Token Index: 151

Text #12

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.9073. Min Act: -0.1700

Data Index: 4123278 (Open Web Text)

Max Activating Token Index: 80

Text #13

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.9085. Min Act: -0.1700

Data Index: 570746 (Open Web Text)

Max Activating Token Index: 119

Text #14

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.9026. Min Act: -0.1700

Data Index: 4988818 (Open Web Text)

Max Activating Token Index: 14

Text #15

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.9024. Min Act: -0.1700

Data Index: 1955569 (Open Web Text)

Max Activating Token Index: 99

Text #16

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.8956. Min Act: -0.1700

Data Index: 2966233 (Open Web Text)

Max Activating Token Index: 300

Text #17

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.8879. Min Act: -0.1700

Data Index: 8459392 (Open Web Text)

Max Activating Token Index: 112

Text #18

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.8916. Min Act: -0.1700

Data Index: 3628326 (Open Web Text)

Max Activating Token Index: 806

Text #19

Max Range: 2.0352. Min Range: -2.0352

Max Act: 1.8838. Min Act: -0.1700

Data Index: 3156102 (Open Web Text)

Max Activating Token Index: 50