Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2192 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6693. Min Act: -0.1682

Data Index: 4381912 (Open Web Text)

Max Activating Token Index: 627

Text #1

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6432. Min Act: -0.0961

Data Index: 3405472 (Open Web Text)

Max Activating Token Index: 698

Text #2

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6273. Min Act: -0.1137

Data Index: 2557320 (Open Web Text)

Max Activating Token Index: 847

Text #3

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6348. Min Act: -0.0878

Data Index: 120686 (Open Web Text)

Max Activating Token Index: 598

Text #4

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6302. Min Act: -0.0899

Data Index: 6442940 (Open Web Text)

Max Activating Token Index: 404

Text #5

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6313. Min Act: -0.0702

Data Index: 8272095 (Open Web Text)

Max Activating Token Index: 152

Text #6

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6312. Min Act: -0.1216

Data Index: 1618731 (Open Web Text)

Max Activating Token Index: 665

Text #7

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6335. Min Act: -0.1311

Data Index: 5323008 (Open Web Text)

Max Activating Token Index: 652

Text #8

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6339. Min Act: -0.1041

Data Index: 1404490 (Open Web Text)

Max Activating Token Index: 609

Text #9

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6328. Min Act: -0.1279

Data Index: 5123386 (Open Web Text)

Max Activating Token Index: 543

Text #10

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6261. Min Act: -0.1248

Data Index: 8244073 (Open Web Text)

Max Activating Token Index: 812

Text #11

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6229. Min Act: -0.1070

Data Index: 8595621 (Open Web Text)

Max Activating Token Index: 871

Text #12

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6253. Min Act: -0.0736

Data Index: 7983129 (Open Web Text)

Max Activating Token Index: 479

Text #13

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6199. Min Act: -0.0578

Data Index: 8831996 (Open Web Text)

Max Activating Token Index: 838

Text #14

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6213. Min Act: -0.0458

Data Index: 6581862 (Open Web Text)

Max Activating Token Index: 672

Text #15

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6282. Min Act: -0.0648

Data Index: 6662585 (Open Web Text)

Max Activating Token Index: 852

Text #16

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6197. Min Act: -0.0886

Data Index: 597220 (Open Web Text)

Max Activating Token Index: 489

Text #17

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6148. Min Act: -0.1173

Data Index: 2205502 (Open Web Text)

Max Activating Token Index: 488

Text #18

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6168. Min Act: -0.1548

Data Index: 1324722 (Open Web Text)

Max Activating Token Index: 502

Text #19

Max Range: 1.6693. Min Range: -1.6693

Max Act: 1.6156. Min Act: -0.0794

Data Index: 4880719 (Open Web Text)

Max Activating Token Index: 822