Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1642 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.8778. Min Act: -0.1700

Data Index: 5687952 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.8400. Min Act: -0.1700

Data Index: 1784379 (Open Web Text)

Max Activating Token Index: 8

Text #2

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.7166. Min Act: -0.1700

Data Index: 6189255 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.6619. Min Act: -0.1700

Data Index: 2450044 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.6025. Min Act: -0.1700

Data Index: 1867493 (Open Web Text)

Max Activating Token Index: 11

Text #5

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5955. Min Act: -0.1700

Data Index: 6061660 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5621. Min Act: -0.1700

Data Index: 3937244 (Open Web Text)

Max Activating Token Index: 7

Text #7

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5554. Min Act: -0.1700

Data Index: 3990029 (Open Web Text)

Max Activating Token Index: 84

Text #8

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5206. Min Act: -0.1700

Data Index: 1313200 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5324. Min Act: -0.1700

Data Index: 3225474 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5161. Min Act: -0.1700

Data Index: 5340476 (Open Web Text)

Max Activating Token Index: 6

Text #11

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5224. Min Act: -0.1700

Data Index: 2672947 (Open Web Text)

Max Activating Token Index: 34

Text #12

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5221. Min Act: -0.1700

Data Index: 1282163 (Open Web Text)

Max Activating Token Index: 920

Text #13

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5131. Min Act: -0.1700

Data Index: 7733819 (Open Web Text)

Max Activating Token Index: 6

Text #14

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5414. Min Act: -0.1700

Data Index: 1773302 (Open Web Text)

Max Activating Token Index: 436

Text #15

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5249. Min Act: -0.1700

Data Index: 6361648 (Open Web Text)

Max Activating Token Index: 7

Text #16

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5179. Min Act: -0.1700

Data Index: 6311232 (Open Web Text)

Max Activating Token Index: 68

Text #17

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.5068. Min Act: -0.1700

Data Index: 120136 (Open Web Text)

Max Activating Token Index: 524

Text #18

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.4913. Min Act: -0.1700

Data Index: 5470708 (Open Web Text)

Max Activating Token Index: 182

Text #19

Max Range: 4.8778. Min Range: -4.8778

Max Act: 4.4784. Min Act: -0.1700

Data Index: 3838270 (Open Web Text)

Max Activating Token Index: 546