Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 520 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0547. Min Range: -3.0547

Max Act: 3.0547. Min Act: -0.1700

Data Index: 4257717 (Open Web Text)

Max Activating Token Index: 447

Text #1

Max Range: 3.0547. Min Range: -3.0547

Max Act: 3.0062. Min Act: -0.1700

Data Index: 3969000 (Open Web Text)

Max Activating Token Index: 254

Text #2

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.8996. Min Act: -0.1700

Data Index: 4667629 (Open Web Text)

Max Activating Token Index: 140

Text #3

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.8396. Min Act: -0.1700

Data Index: 2069366 (Open Web Text)

Max Activating Token Index: 547

Text #4

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7978. Min Act: -0.1700

Data Index: 5922039 (Open Web Text)

Max Activating Token Index: 200

Text #5

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7844. Min Act: -0.1700

Data Index: 7567801 (Open Web Text)

Max Activating Token Index: 141

Text #6

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7539. Min Act: -0.1700

Data Index: 7826571 (Open Web Text)

Max Activating Token Index: 109

Text #7

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7700. Min Act: -0.1700

Data Index: 5740746 (Open Web Text)

Max Activating Token Index: 452

Text #8

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7500. Min Act: -0.1700

Data Index: 5658265 (Open Web Text)

Max Activating Token Index: 185

Text #9

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7603. Min Act: -0.1700

Data Index: 5193521 (Open Web Text)

Max Activating Token Index: 60

Text #10

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7430. Min Act: -0.1700

Data Index: 5793702 (Open Web Text)

Max Activating Token Index: 39

Text #11

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7625. Min Act: -0.1700

Data Index: 6088927 (Open Web Text)

Max Activating Token Index: 92

Text #12

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7479. Min Act: -0.1700

Data Index: 1833350 (Open Web Text)

Max Activating Token Index: 88

Text #13

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7363. Min Act: -0.1700

Data Index: 5037936 (Open Web Text)

Max Activating Token Index: 90

Text #14

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7366. Min Act: -0.1700

Data Index: 776686 (Open Web Text)

Max Activating Token Index: 162

Text #15

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.6837. Min Act: -0.1700

Data Index: 1082742 (Open Web Text)

Max Activating Token Index: 135

Text #16

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.6877. Min Act: -0.1700

Data Index: 5137297 (Open Web Text)

Max Activating Token Index: 30

Text #17

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7045. Min Act: -0.1700

Data Index: 607077 (Open Web Text)

Max Activating Token Index: 90

Text #18

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.6830. Min Act: -0.1700

Data Index: 3137239 (Open Web Text)

Max Activating Token Index: 71

Text #19

Max Range: 3.0547. Min Range: -3.0547

Max Act: 2.7037. Min Act: -0.1700

Data Index: 8794983 (Open Web Text)

Max Activating Token Index: 574