Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1389 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.4404. Min Act: -0.1700

Data Index: 5319244 (Open Web Text)

Max Activating Token Index: 15

Text #1

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.3216. Min Act: -0.1700

Data Index: 771597 (Open Web Text)

Max Activating Token Index: 165

Text #2

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.2636. Min Act: -0.1700

Data Index: 8101803 (Open Web Text)

Max Activating Token Index: 228

Text #3

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.2862. Min Act: -0.1700

Data Index: 2776405 (Open Web Text)

Max Activating Token Index: 890

Text #4

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.2524. Min Act: -0.1700

Data Index: 7333601 (Open Web Text)

Max Activating Token Index: 359

Text #5

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.2648. Min Act: -0.1700

Data Index: 1117549 (Open Web Text)

Max Activating Token Index: 129

Text #6

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.2337. Min Act: -0.1700

Data Index: 5645663 (Open Web Text)

Max Activating Token Index: 1019

Text #7

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.2290. Min Act: -0.1700

Data Index: 5530746 (Open Web Text)

Max Activating Token Index: 787

Text #8

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.2481. Min Act: -0.1700

Data Index: 8608019 (Open Web Text)

Max Activating Token Index: 27

Text #9

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.2240. Min Act: -0.1700

Data Index: 8101809 (Open Web Text)

Max Activating Token Index: 60

Text #10

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.2390. Min Act: -0.1700

Data Index: 1573046 (Open Web Text)

Max Activating Token Index: 14

Text #11

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.1818. Min Act: -0.1700

Data Index: 4959017 (Open Web Text)

Max Activating Token Index: 804

Text #12

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.1917. Min Act: -0.1700

Data Index: 8612510 (Open Web Text)

Max Activating Token Index: 741

Text #13

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.1763. Min Act: -0.1700

Data Index: 5645666 (Open Web Text)

Max Activating Token Index: 25

Text #14

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.1918. Min Act: -0.1700

Data Index: 45473 (Open Web Text)

Max Activating Token Index: 48

Text #15

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.1910. Min Act: -0.1700

Data Index: 1536296 (Open Web Text)

Max Activating Token Index: 835

Text #16

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.2268. Min Act: -0.1700

Data Index: 3649279 (Open Web Text)

Max Activating Token Index: 448

Text #17

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.1545. Min Act: -0.1700

Data Index: 3510424 (Open Web Text)

Max Activating Token Index: 671

Text #18

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.1827. Min Act: -0.1700

Data Index: 5567177 (Open Web Text)

Max Activating Token Index: 669

Text #19

Max Range: 4.4404. Min Range: -4.4404

Max Act: 4.1618. Min Act: -0.1700

Data Index: 232863 (Open Web Text)

Max Activating Token Index: 490