Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2808 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.3274. Min Range: -6.3274

Max Act: 6.3274. Min Act: -0.1700

Data Index: 3888892 (Open Web Text)

Max Activating Token Index: 186

Text #1

Max Range: 6.3274. Min Range: -6.3274

Max Act: 6.0212. Min Act: -0.1700

Data Index: 7543650 (Open Web Text)

Max Activating Token Index: 609

Text #2

Max Range: 6.3274. Min Range: -6.3274

Max Act: 6.0818. Min Act: -0.1700

Data Index: 7472299 (Open Web Text)

Max Activating Token Index: 488

Text #3

Max Range: 6.3274. Min Range: -6.3274

Max Act: 6.0375. Min Act: -0.1700

Data Index: 5576102 (Open Web Text)

Max Activating Token Index: 411

Text #4

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.9656. Min Act: -0.1700

Data Index: 3983014 (Open Web Text)

Max Activating Token Index: 289

Text #5

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.9242. Min Act: -0.1700

Data Index: 8203076 (Open Web Text)

Max Activating Token Index: 890

Text #6

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.9374. Min Act: -0.1700

Data Index: 4556011 (Open Web Text)

Max Activating Token Index: 522

Text #7

Max Range: 6.3274. Min Range: -6.3274

Max Act: 6.0320. Min Act: -0.1700

Data Index: 4725880 (Open Web Text)

Max Activating Token Index: 884

Text #8

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.9592. Min Act: -0.1700

Data Index: 1299329 (Open Web Text)

Max Activating Token Index: 365

Text #9

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.8906. Min Act: -0.1700

Data Index: 325450 (Open Web Text)

Max Activating Token Index: 928

Text #10

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.9106. Min Act: -0.1700

Data Index: 3834347 (Open Web Text)

Max Activating Token Index: 73

Text #11

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.9511. Min Act: -0.1700

Data Index: 5402809 (Open Web Text)

Max Activating Token Index: 975

Text #12

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.9255. Min Act: -0.1700

Data Index: 4994934 (Open Web Text)

Max Activating Token Index: 851

Text #13

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.8687. Min Act: -0.1700

Data Index: 3065019 (Open Web Text)

Max Activating Token Index: 928

Text #14

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.9224. Min Act: -0.1700

Data Index: 1992794 (Open Web Text)

Max Activating Token Index: 366

Text #15

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.9687. Min Act: -0.1700

Data Index: 3039773 (Open Web Text)

Max Activating Token Index: 913

Text #16

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.9365. Min Act: -0.1700

Data Index: 15389 (Open Web Text)

Max Activating Token Index: 1002

Text #17

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.8954. Min Act: -0.1700

Data Index: 8565656 (Open Web Text)

Max Activating Token Index: 479

Text #18

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.9210. Min Act: -0.1700

Data Index: 4052737 (Open Web Text)

Max Activating Token Index: 499

Text #19

Max Range: 6.3274. Min Range: -6.3274

Max Act: 5.8471. Min Act: -0.1700

Data Index: 5158540 (Open Web Text)

Max Activating Token Index: 431