Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2410 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.2229. Min Range: -6.2229

Max Act: 6.2229. Min Act: -0.1700

Data Index: 3552897 (Open Web Text)

Max Activating Token Index: 852

Text #1

Max Range: 6.2229. Min Range: -6.2229

Max Act: 6.1431. Min Act: -0.1700

Data Index: 2748391 (Open Web Text)

Max Activating Token Index: 64

Text #2

Max Range: 6.2229. Min Range: -6.2229

Max Act: 6.0378. Min Act: -0.1700

Data Index: 7603132 (Open Web Text)

Max Activating Token Index: 277

Text #3

Max Range: 6.2229. Min Range: -6.2229

Max Act: 6.0628. Min Act: -0.1700

Data Index: 8565211 (Open Web Text)

Max Activating Token Index: 864

Text #4

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.9399. Min Act: -0.1700

Data Index: 6842157 (Open Web Text)

Max Activating Token Index: 936

Text #5

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.9440. Min Act: -0.1700

Data Index: 2506930 (Open Web Text)

Max Activating Token Index: 71

Text #6

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.9506. Min Act: -0.1700

Data Index: 6245913 (Open Web Text)

Max Activating Token Index: 68

Text #7

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.8906. Min Act: -0.1700

Data Index: 4403752 (Open Web Text)

Max Activating Token Index: 413

Text #8

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.9320. Min Act: -0.1700

Data Index: 551318 (Open Web Text)

Max Activating Token Index: 291

Text #9

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.9206. Min Act: -0.1700

Data Index: 4067686 (Open Web Text)

Max Activating Token Index: 549

Text #10

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.9270. Min Act: -0.1700

Data Index: 8070230 (Open Web Text)

Max Activating Token Index: 318

Text #11

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.8808. Min Act: -0.1700

Data Index: 5566996 (Open Web Text)

Max Activating Token Index: 658

Text #12

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.9117. Min Act: -0.1700

Data Index: 1390573 (Open Web Text)

Max Activating Token Index: 933

Text #13

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.8447. Min Act: -0.1700

Data Index: 7844090 (Open Web Text)

Max Activating Token Index: 199

Text #14

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.8751. Min Act: -0.1700

Data Index: 6799211 (Open Web Text)

Max Activating Token Index: 431

Text #15

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.8635. Min Act: -0.1700

Data Index: 5835041 (Open Web Text)

Max Activating Token Index: 255

Text #16

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.8353. Min Act: -0.1700

Data Index: 762034 (Open Web Text)

Max Activating Token Index: 794

Text #17

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.8548. Min Act: -0.1700

Data Index: 4126565 (Open Web Text)

Max Activating Token Index: 374

Text #18

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.8342. Min Act: -0.1700

Data Index: 4700865 (Open Web Text)

Max Activating Token Index: 612

Text #19

Max Range: 6.2229. Min Range: -6.2229

Max Act: 5.8429. Min Act: -0.1700

Data Index: 2748392 (Open Web Text)

Max Activating Token Index: 368