Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.7315. Min Act: -0.1699

Data Index: 1114735 (Open Web Text)

Max Activating Token Index: 161

Text #1

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.7202. Min Act: -0.1700

Data Index: 6633852 (Open Web Text)

Max Activating Token Index: 297

Text #2

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.6687. Min Act: -0.1700

Data Index: 8219767 (Open Web Text)

Max Activating Token Index: 164

Text #3

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.6925. Min Act: -0.1698

Data Index: 7990286 (Open Web Text)

Max Activating Token Index: 92

Text #4

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.6648. Min Act: -0.1700

Data Index: 215121 (Open Web Text)

Max Activating Token Index: 539

Text #5

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.6470. Min Act: -0.1700

Data Index: 7727752 (Open Web Text)

Max Activating Token Index: 245

Text #6

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.6296. Min Act: -0.1700

Data Index: 1227617 (Open Web Text)

Max Activating Token Index: 737

Text #7

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.6211. Min Act: -0.1700

Data Index: 6245007 (Open Web Text)

Max Activating Token Index: 202

Text #8

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.6534. Min Act: -0.1700

Data Index: 8504456 (Open Web Text)

Max Activating Token Index: 323

Text #9

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.5749. Min Act: -0.1699

Data Index: 4767698 (Open Web Text)

Max Activating Token Index: 627

Text #10

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.5795. Min Act: -0.1700

Data Index: 4079270 (Open Web Text)

Max Activating Token Index: 251

Text #11

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.5912. Min Act: -0.1700

Data Index: 7576566 (Open Web Text)

Max Activating Token Index: 133

Text #12

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.5732. Min Act: -0.1700

Data Index: 6837449 (Open Web Text)

Max Activating Token Index: 381

Text #13

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.5715. Min Act: -0.1700

Data Index: 1935041 (Open Web Text)

Max Activating Token Index: 179

Text #14

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.5740. Min Act: -0.1700

Data Index: 542622 (Open Web Text)

Max Activating Token Index: 575

Text #15

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.5417. Min Act: -0.1700

Data Index: 371158 (Open Web Text)

Max Activating Token Index: 675

Text #16

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.5190. Min Act: -0.1700

Data Index: 7298253 (Open Web Text)

Max Activating Token Index: 391

Text #17

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.5123. Min Act: -0.1700

Data Index: 5603510 (Open Web Text)

Max Activating Token Index: 765

Text #18

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.5390. Min Act: -0.1700

Data Index: 2181707 (Open Web Text)

Max Activating Token Index: 336

Text #19

Max Range: 5.7315. Min Range: -5.7315

Max Act: 5.5447. Min Act: -0.1700

Data Index: 2563528 (Open Web Text)

Max Activating Token Index: 64