Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 679 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.9913. Min Act: -0.1700

Data Index: 7142910 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.9481. Min Act: -0.1700

Data Index: 8233781 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.9546. Min Act: -0.1700

Data Index: 851364 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.9057. Min Act: -0.1700

Data Index: 3311199 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.9187. Min Act: -0.1700

Data Index: 8325980 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.9234. Min Act: -0.1700

Data Index: 5156896 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.9019. Min Act: -0.1700

Data Index: 1858283 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8822. Min Act: -0.1700

Data Index: 47556 (Open Web Text)

Max Activating Token Index: 7

Text #8

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8912. Min Act: -0.1700

Data Index: 3964702 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8542. Min Act: -0.1700

Data Index: 7493967 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8445. Min Act: -0.1700

Data Index: 504485 (Open Web Text)

Max Activating Token Index: 10

Text #11

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8696. Min Act: -0.1700

Data Index: 150258 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8533. Min Act: -0.1700

Data Index: 7539556 (Open Web Text)

Max Activating Token Index: 8

Text #13

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8437. Min Act: -0.1700

Data Index: 420552 (Open Web Text)

Max Activating Token Index: 8

Text #14

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8200. Min Act: -0.1700

Data Index: 4919843 (Open Web Text)

Max Activating Token Index: 14

Text #15

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8127. Min Act: -0.1700

Data Index: 4865074 (Open Web Text)

Max Activating Token Index: 701

Text #16

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8267. Min Act: -0.1700

Data Index: 945236 (Open Web Text)

Max Activating Token Index: 13

Text #17

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8142. Min Act: -0.1700

Data Index: 6923341 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8194. Min Act: -0.1700

Data Index: 7346384 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 3.9913. Min Range: -3.9913

Max Act: 3.8396. Min Act: -0.1700

Data Index: 2620880 (Open Web Text)

Max Activating Token Index: 4