Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1123 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.9340. Min Act: -0.1700

Data Index: 4852395 (Open Web Text)

Max Activating Token Index: 31

Text #1

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8894. Min Act: -0.1700

Data Index: 188612 (Open Web Text)

Max Activating Token Index: 114

Text #2

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8529. Min Act: -0.1700

Data Index: 7345883 (Open Web Text)

Max Activating Token Index: 424

Text #3

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8535. Min Act: -0.1700

Data Index: 7734243 (Open Web Text)

Max Activating Token Index: 622

Text #4

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8237. Min Act: -0.1700

Data Index: 1116273 (Open Web Text)

Max Activating Token Index: 484

Text #5

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8202. Min Act: -0.1700

Data Index: 1359317 (Open Web Text)

Max Activating Token Index: 665

Text #6

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8234. Min Act: -0.1700

Data Index: 3465881 (Open Web Text)

Max Activating Token Index: 344

Text #7

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8247. Min Act: -0.1700

Data Index: 5008813 (Open Web Text)

Max Activating Token Index: 112

Text #8

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8256. Min Act: -0.1700

Data Index: 6420497 (Open Web Text)

Max Activating Token Index: 620

Text #9

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8304. Min Act: -0.1700

Data Index: 6968441 (Open Web Text)

Max Activating Token Index: 35

Text #10

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8294. Min Act: -0.1700

Data Index: 6874137 (Open Web Text)

Max Activating Token Index: 358

Text #11

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8128. Min Act: -0.1700

Data Index: 6735460 (Open Web Text)

Max Activating Token Index: 271

Text #12

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8337. Min Act: -0.1700

Data Index: 3916215 (Open Web Text)

Max Activating Token Index: 325

Text #13

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.7944. Min Act: -0.1700

Data Index: 1098512 (Open Web Text)

Max Activating Token Index: 162

Text #14

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.7975. Min Act: -0.1700

Data Index: 2781340 (Open Web Text)

Max Activating Token Index: 598

Text #15

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8056. Min Act: -0.1700

Data Index: 5360683 (Open Web Text)

Max Activating Token Index: 580

Text #16

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8118. Min Act: -0.1700

Data Index: 5688863 (Open Web Text)

Max Activating Token Index: 494

Text #17

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8139. Min Act: -0.1700

Data Index: 5724346 (Open Web Text)

Max Activating Token Index: 267

Text #18

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8030. Min Act: -0.1700

Data Index: 3694650 (Open Web Text)

Max Activating Token Index: 293

Text #19

Max Range: 4.9340. Min Range: -4.9340

Max Act: 4.8128. Min Act: -0.1700

Data Index: 1423830 (Open Web Text)

Max Activating Token Index: 445