Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 120 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.8010. Min Act: -0.1700

Data Index: 5478144 (Open Web Text)

Max Activating Token Index: 19

Text #1

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.8157. Min Act: -0.1700

Data Index: 8828317 (Open Web Text)

Max Activating Token Index: 683

Text #2

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.7285. Min Act: -0.1700

Data Index: 8503911 (Open Web Text)

Max Activating Token Index: 120

Text #3

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.7266. Min Act: -0.1700

Data Index: 2885981 (Open Web Text)

Max Activating Token Index: 161

Text #4

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.7223. Min Act: -0.1700

Data Index: 6428837 (Open Web Text)

Max Activating Token Index: 23

Text #5

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6720. Min Act: -0.1700

Data Index: 1319446 (Open Web Text)

Max Activating Token Index: 928

Text #6

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6898. Min Act: -0.1700

Data Index: 1488958 (Open Web Text)

Max Activating Token Index: 26

Text #7

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6489. Min Act: -0.1700

Data Index: 7054358 (Open Web Text)

Max Activating Token Index: 39

Text #8

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6880. Min Act: -0.1700

Data Index: 4966247 (Open Web Text)

Max Activating Token Index: 979

Text #9

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6841. Min Act: -0.1700

Data Index: 4321738 (Open Web Text)

Max Activating Token Index: 73

Text #10

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6856. Min Act: -0.1700

Data Index: 4953485 (Open Web Text)

Max Activating Token Index: 33

Text #11

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6261. Min Act: -0.1700

Data Index: 2845110 (Open Web Text)

Max Activating Token Index: 15

Text #12

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6422. Min Act: -0.1700

Data Index: 4711989 (Open Web Text)

Max Activating Token Index: 442

Text #13

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6218. Min Act: -0.1700

Data Index: 5016766 (Open Web Text)

Max Activating Token Index: 142

Text #14

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6274. Min Act: -0.1700

Data Index: 4620409 (Open Web Text)

Max Activating Token Index: 58

Text #15

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6473. Min Act: -0.1700

Data Index: 8768277 (Open Web Text)

Max Activating Token Index: 43

Text #16

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6200. Min Act: -0.1700

Data Index: 5057091 (Open Web Text)

Max Activating Token Index: 548

Text #17

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6543. Min Act: -0.1700

Data Index: 7987175 (Open Web Text)

Max Activating Token Index: 62

Text #18

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6515. Min Act: -0.1700

Data Index: 45504 (Open Web Text)

Max Activating Token Index: 751

Text #19

Max Range: 3.8157. Min Range: -3.8157

Max Act: 3.6199. Min Act: -0.1700

Data Index: 2050160 (Open Web Text)

Max Activating Token Index: 128