Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 759 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.7301. Min Act: -0.0994

Data Index: 4908111 (Open Web Text)

Max Activating Token Index: 412

Text #1

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.7224. Min Act: -0.0948

Data Index: 47052 (Open Web Text)

Max Activating Token Index: 131

Text #2

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.7001. Min Act: -0.1251

Data Index: 1751693 (Open Web Text)

Max Activating Token Index: 867

Text #3

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6962. Min Act: -0.0899

Data Index: 8167258 (Open Web Text)

Max Activating Token Index: 170

Text #4

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.7029. Min Act: -0.1089

Data Index: 5311913 (Open Web Text)

Max Activating Token Index: 624

Text #5

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.7017. Min Act: -0.0286

Data Index: 8777796 (Open Web Text)

Max Activating Token Index: 406

Text #6

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6754. Min Act: -0.1308

Data Index: 2218381 (Open Web Text)

Max Activating Token Index: 1018

Text #7

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6694. Min Act: -0.0363

Data Index: 3191631 (Open Web Text)

Max Activating Token Index: 258

Text #8

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6686. Min Act: -0.1245

Data Index: 3571538 (Open Web Text)

Max Activating Token Index: 362

Text #9

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6654. Min Act: -0.0945

Data Index: 3119134 (Open Web Text)

Max Activating Token Index: 102

Text #10

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6537. Min Act: -0.0655

Data Index: 3459078 (Open Web Text)

Max Activating Token Index: 1013

Text #11

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6321. Min Act: -0.0924

Data Index: 7293847 (Open Web Text)

Max Activating Token Index: 97

Text #12

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6423. Min Act: -0.1229

Data Index: 1417018 (Open Web Text)

Max Activating Token Index: 949

Text #13

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6302. Min Act: -0.0879

Data Index: 5888513 (Open Web Text)

Max Activating Token Index: 146

Text #14

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6333. Min Act: -0.0484

Data Index: 6378876 (Open Web Text)

Max Activating Token Index: 532

Text #15

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6249. Min Act: -0.0564

Data Index: 4444713 (Open Web Text)

Max Activating Token Index: 583

Text #16

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6273. Min Act: -0.1309

Data Index: 4958868 (Open Web Text)

Max Activating Token Index: 607

Text #17

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6327. Min Act: -0.0351

Data Index: 2575356 (Open Web Text)

Max Activating Token Index: 1010

Text #18

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6136. Min Act: -0.0003

Data Index: 4338094 (Open Web Text)

Max Activating Token Index: 950

Text #19

Max Range: 1.7301. Min Range: -1.7301

Max Act: 1.6023. Min Act: -0.0216

Data Index: 2333260 (Open Web Text)

Max Activating Token Index: 830