Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 837 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.9213. Min Act: -0.1700

Data Index: 6933893 (Open Web Text)

Max Activating Token Index: 6

Text #1

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.9115. Min Act: -0.1700

Data Index: 7037289 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8821. Min Act: -0.1700

Data Index: 6276110 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8471. Min Act: -0.1700

Data Index: 2502131 (Open Web Text)

Max Activating Token Index: 8

Text #4

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8359. Min Act: -0.1700

Data Index: 3351140 (Open Web Text)

Max Activating Token Index: 5

Text #5

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8432. Min Act: -0.1700

Data Index: 4106389 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8404. Min Act: -0.1700

Data Index: 5629079 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8325. Min Act: -0.1700

Data Index: 1834997 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8315. Min Act: -0.1700

Data Index: 2823687 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8266. Min Act: -0.1700

Data Index: 788663 (Open Web Text)

Max Activating Token Index: 7

Text #10

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8290. Min Act: -0.1700

Data Index: 1647698 (Open Web Text)

Max Activating Token Index: 372

Text #11

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8117. Min Act: -0.1700

Data Index: 5530940 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8053. Min Act: -0.1700

Data Index: 7097826 (Open Web Text)

Max Activating Token Index: 122

Text #13

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8035. Min Act: -0.1700

Data Index: 1523987 (Open Web Text)

Max Activating Token Index: 10

Text #14

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8127. Min Act: -0.1700

Data Index: 6508412 (Open Web Text)

Max Activating Token Index: 8

Text #15

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8119. Min Act: -0.1700

Data Index: 353187 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8101. Min Act: -0.1700

Data Index: 7274741 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.8110. Min Act: -0.1700

Data Index: 1706240 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.7929. Min Act: -0.1700

Data Index: 205695 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 2.9213. Min Range: -2.9213

Max Act: 2.7861. Min Act: -0.1700

Data Index: 5475305 (Open Web Text)

Max Activating Token Index: 3