Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2665 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.5267. Min Act: -0.1278

Data Index: 5823864 (Open Web Text)

Max Activating Token Index: 1007

Text #1

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4747. Min Act: -0.1278

Data Index: 3456012 (Open Web Text)

Max Activating Token Index: 978

Text #2

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4868. Min Act: -0.1278

Data Index: 6382236 (Open Web Text)

Max Activating Token Index: 138

Text #3

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4486. Min Act: -0.1278

Data Index: 4421942 (Open Web Text)

Max Activating Token Index: 1022

Text #4

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4569. Min Act: -0.1278

Data Index: 6297854 (Open Web Text)

Max Activating Token Index: 959

Text #5

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4453. Min Act: -0.1278

Data Index: 7716338 (Open Web Text)

Max Activating Token Index: 999

Text #6

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4458. Min Act: -0.1278

Data Index: 4843930 (Open Web Text)

Max Activating Token Index: 991

Text #7

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4451. Min Act: -0.1278

Data Index: 5737214 (Open Web Text)

Max Activating Token Index: 1013

Text #8

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4360. Min Act: -0.1278

Data Index: 4451093 (Open Web Text)

Max Activating Token Index: 1022

Text #9

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4361. Min Act: -0.1278

Data Index: 1240323 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4270. Min Act: -0.1278

Data Index: 5164350 (Open Web Text)

Max Activating Token Index: 1021

Text #11

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4335. Min Act: -0.1278

Data Index: 5195393 (Open Web Text)

Max Activating Token Index: 1008

Text #12

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4348. Min Act: -0.1278

Data Index: 7099245 (Open Web Text)

Max Activating Token Index: 955

Text #13

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4253. Min Act: -0.1278

Data Index: 1759548 (Open Web Text)

Max Activating Token Index: 1001

Text #14

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4337. Min Act: -0.1278

Data Index: 5618325 (Open Web Text)

Max Activating Token Index: 949

Text #15

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4185. Min Act: -0.1278

Data Index: 2071 (Open Web Text)

Max Activating Token Index: 957

Text #16

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4155. Min Act: -0.1278

Data Index: 6784798 (Open Web Text)

Max Activating Token Index: 1014

Text #17

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4178. Min Act: -0.1278

Data Index: 5497533 (Open Web Text)

Max Activating Token Index: 1001

Text #18

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4213. Min Act: -0.1278

Data Index: 3663634 (Open Web Text)

Max Activating Token Index: 689

Text #19

Max Range: 1.5267. Min Range: -1.5267

Max Act: 1.4236. Min Act: -0.1278

Data Index: 7450622 (Open Web Text)

Max Activating Token Index: 91