Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 826 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.7240. Min Act: -0.1700

Data Index: 3776924 (Open Web Text)

Max Activating Token Index: 78

Text #1

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.7237. Min Act: -0.1700

Data Index: 7772049 (Open Web Text)

Max Activating Token Index: 175

Text #2

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.7022. Min Act: -0.1700

Data Index: 6083407 (Open Web Text)

Max Activating Token Index: 492

Text #3

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6861. Min Act: -0.1700

Data Index: 6505300 (Open Web Text)

Max Activating Token Index: 83

Text #4

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.7046. Min Act: -0.1700

Data Index: 7521373 (Open Web Text)

Max Activating Token Index: 41

Text #5

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6652. Min Act: -0.1700

Data Index: 865726 (Open Web Text)

Max Activating Token Index: 606

Text #6

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6456. Min Act: -0.1700

Data Index: 3993769 (Open Web Text)

Max Activating Token Index: 88

Text #7

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6470. Min Act: -0.1700

Data Index: 7714448 (Open Web Text)

Max Activating Token Index: 46

Text #8

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6439. Min Act: -0.1700

Data Index: 6601902 (Open Web Text)

Max Activating Token Index: 58

Text #9

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6278. Min Act: -0.1700

Data Index: 1753233 (Open Web Text)

Max Activating Token Index: 845

Text #10

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6385. Min Act: -0.1700

Data Index: 868823 (Open Web Text)

Max Activating Token Index: 55

Text #11

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6322. Min Act: -0.1700

Data Index: 7568503 (Open Web Text)

Max Activating Token Index: 37

Text #12

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6408. Min Act: -0.1700

Data Index: 492819 (Open Web Text)

Max Activating Token Index: 11

Text #13

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6188. Min Act: -0.1700

Data Index: 8417033 (Open Web Text)

Max Activating Token Index: 23

Text #14

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6024. Min Act: -0.1700

Data Index: 2527179 (Open Web Text)

Max Activating Token Index: 93

Text #15

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6225. Min Act: -0.1700

Data Index: 1699973 (Open Web Text)

Max Activating Token Index: 675

Text #16

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6181. Min Act: -0.1700

Data Index: 7660997 (Open Web Text)

Max Activating Token Index: 58

Text #17

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6165. Min Act: -0.1700

Data Index: 3819571 (Open Web Text)

Max Activating Token Index: 23

Text #18

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6171. Min Act: -0.1700

Data Index: 3385712 (Open Web Text)

Max Activating Token Index: 51

Text #19

Max Range: 1.7240. Min Range: -1.7240

Max Act: 1.6100. Min Act: -0.1700

Data Index: 2394631 (Open Web Text)

Max Activating Token Index: 269