Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1876 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.9058. Min Act: -0.1700

Data Index: 7630190 (Open Web Text)

Max Activating Token Index: 10

Text #1

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8843. Min Act: -0.1700

Data Index: 6451045 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8518. Min Act: -0.1700

Data Index: 5481684 (Open Web Text)

Max Activating Token Index: 8

Text #3

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8596. Min Act: -0.1700

Data Index: 2095250 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8665. Min Act: -0.1700

Data Index: 3772369 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8683. Min Act: -0.1700

Data Index: 6274778 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8670. Min Act: -0.1700

Data Index: 1119260 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8578. Min Act: -0.1700

Data Index: 1434193 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8407. Min Act: -0.1700

Data Index: 2009402 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8437. Min Act: -0.1700

Data Index: 5762434 (Open Web Text)

Max Activating Token Index: 20

Text #10

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8419. Min Act: -0.1700

Data Index: 3924483 (Open Web Text)

Max Activating Token Index: 9

Text #11

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8555. Min Act: -0.1700

Data Index: 2960058 (Open Web Text)

Max Activating Token Index: 4

Text #12

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8252. Min Act: -0.1700

Data Index: 6779126 (Open Web Text)

Max Activating Token Index: 6

Text #13

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8349. Min Act: -0.1700

Data Index: 4412694 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8234. Min Act: -0.1700

Data Index: 7105852 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8214. Min Act: -0.1700

Data Index: 2503444 (Open Web Text)

Max Activating Token Index: 7

Text #16

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8242. Min Act: -0.1700

Data Index: 400420 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8112. Min Act: -0.1700

Data Index: 2781025 (Open Web Text)

Max Activating Token Index: 2

Text #18

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8130. Min Act: -0.1700

Data Index: 4240814 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 2.9058. Min Range: -2.9058

Max Act: 2.8196. Min Act: -0.1700

Data Index: 4166 (Open Web Text)

Max Activating Token Index: 3