Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2388 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.7087. Min Act: -0.1700

Data Index: 6115280 (Open Web Text)

Max Activating Token Index: 165

Text #1

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6878. Min Act: -0.1700

Data Index: 254183 (Open Web Text)

Max Activating Token Index: 923

Text #2

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.7151. Min Act: -0.1700

Data Index: 7294470 (Open Web Text)

Max Activating Token Index: 874

Text #3

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6489. Min Act: -0.1700

Data Index: 5671253 (Open Web Text)

Max Activating Token Index: 454

Text #4

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6551. Min Act: -0.1700

Data Index: 5761839 (Open Web Text)

Max Activating Token Index: 645

Text #5

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6425. Min Act: -0.1700

Data Index: 5990895 (Open Web Text)

Max Activating Token Index: 127

Text #6

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6468. Min Act: -0.1700

Data Index: 1337919 (Open Web Text)

Max Activating Token Index: 493

Text #7

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6335. Min Act: -0.1700

Data Index: 8237694 (Open Web Text)

Max Activating Token Index: 267

Text #8

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6567. Min Act: -0.1700

Data Index: 1337916 (Open Web Text)

Max Activating Token Index: 469

Text #9

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6151. Min Act: -0.1700

Data Index: 3588508 (Open Web Text)

Max Activating Token Index: 549

Text #10

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6235. Min Act: -0.1700

Data Index: 8365088 (Open Web Text)

Max Activating Token Index: 318

Text #11

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6254. Min Act: -0.1700

Data Index: 1837454 (Open Web Text)

Max Activating Token Index: 664

Text #12

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6156. Min Act: -0.1700

Data Index: 7228127 (Open Web Text)

Max Activating Token Index: 100

Text #13

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6379. Min Act: -0.1700

Data Index: 4055278 (Open Web Text)

Max Activating Token Index: 298

Text #14

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6273. Min Act: -0.1700

Data Index: 1570516 (Open Web Text)

Max Activating Token Index: 503

Text #15

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6242. Min Act: -0.1700

Data Index: 7144150 (Open Web Text)

Max Activating Token Index: 133

Text #16

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6342. Min Act: -0.1700

Data Index: 6545531 (Open Web Text)

Max Activating Token Index: 766

Text #17

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6256. Min Act: -0.1700

Data Index: 6433665 (Open Web Text)

Max Activating Token Index: 287

Text #18

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6343. Min Act: -0.1700

Data Index: 6270935 (Open Web Text)

Max Activating Token Index: 624

Text #19

Max Range: 3.7151. Min Range: -3.7151

Max Act: 3.6162. Min Act: -0.1700

Data Index: 2881399 (Open Web Text)

Max Activating Token Index: 94