Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1803 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.9573. Min Act: -0.1700

Data Index: 2382379 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.9267. Min Act: -0.1700

Data Index: 1689483 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8877. Min Act: -0.1700

Data Index: 2559897 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8714. Min Act: -0.1700

Data Index: 8232364 (Open Web Text)

Max Activating Token Index: 10

Text #4

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8641. Min Act: -0.1700

Data Index: 2824620 (Open Web Text)

Max Activating Token Index: 7

Text #5

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8236. Min Act: -0.1700

Data Index: 7608412 (Open Web Text)

Max Activating Token Index: 9

Text #6

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8232. Min Act: -0.1700

Data Index: 6239252 (Open Web Text)

Max Activating Token Index: 9

Text #7

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8273. Min Act: -0.1700

Data Index: 257192 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8119. Min Act: -0.1700

Data Index: 6000930 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8421. Min Act: -0.1700

Data Index: 943318 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8214. Min Act: -0.1700

Data Index: 2228873 (Open Web Text)

Max Activating Token Index: 8

Text #11

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8123. Min Act: -0.1700

Data Index: 21167 (Open Web Text)

Max Activating Token Index: 4

Text #12

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8516. Min Act: -0.1700

Data Index: 4059775 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8404. Min Act: -0.1700

Data Index: 3502960 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8177. Min Act: -0.1700

Data Index: 6599527 (Open Web Text)

Max Activating Token Index: 7

Text #15

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8045. Min Act: -0.1700

Data Index: 4125460 (Open Web Text)

Max Activating Token Index: 5

Text #16

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.7848. Min Act: -0.1700

Data Index: 5263523 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8078. Min Act: -0.1700

Data Index: 3566860 (Open Web Text)

Max Activating Token Index: 3

Text #18

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.8063. Min Act: -0.1700

Data Index: 2857406 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 5.9573. Min Range: -5.9573

Max Act: 5.7844. Min Act: -0.1700

Data Index: 3036010 (Open Web Text)

Max Activating Token Index: 4