Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 608 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.8720. Min Act: -0.1700

Data Index: 4906048 (Open Web Text)

Max Activating Token Index: 427

Text #1

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.7320. Min Act: -0.1700

Data Index: 8076026 (Open Web Text)

Max Activating Token Index: 1010

Text #2

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.6604. Min Act: -0.1700

Data Index: 3505525 (Open Web Text)

Max Activating Token Index: 109

Text #3

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.6342. Min Act: -0.1700

Data Index: 6995552 (Open Web Text)

Max Activating Token Index: 484

Text #4

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.6137. Min Act: -0.1700

Data Index: 3357484 (Open Web Text)

Max Activating Token Index: 294

Text #5

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.6143. Min Act: -0.1700

Data Index: 4984267 (Open Web Text)

Max Activating Token Index: 364

Text #6

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.5945. Min Act: -0.1700

Data Index: 8671655 (Open Web Text)

Max Activating Token Index: 478

Text #7

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.6034. Min Act: -0.1700

Data Index: 1104206 (Open Web Text)

Max Activating Token Index: 124

Text #8

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.5739. Min Act: -0.1700

Data Index: 426317 (Open Web Text)

Max Activating Token Index: 752

Text #9

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.6391. Min Act: -0.1700

Data Index: 6703849 (Open Web Text)

Max Activating Token Index: 168

Text #10

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.5843. Min Act: -0.1700

Data Index: 6070333 (Open Web Text)

Max Activating Token Index: 656

Text #11

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.5763. Min Act: -0.1700

Data Index: 7901349 (Open Web Text)

Max Activating Token Index: 603

Text #12

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.5323. Min Act: -0.1700

Data Index: 7833863 (Open Web Text)

Max Activating Token Index: 378

Text #13

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.5588. Min Act: -0.1700

Data Index: 1571169 (Open Web Text)

Max Activating Token Index: 806

Text #14

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.5670. Min Act: -0.1700

Data Index: 1335900 (Open Web Text)

Max Activating Token Index: 928

Text #15

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.5578. Min Act: -0.1700

Data Index: 6104822 (Open Web Text)

Max Activating Token Index: 526

Text #16

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.5333. Min Act: -0.1700

Data Index: 5764058 (Open Web Text)

Max Activating Token Index: 161

Text #17

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.5311. Min Act: -0.1700

Data Index: 4219851 (Open Web Text)

Max Activating Token Index: 942

Text #18

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.5190. Min Act: -0.1700

Data Index: 652523 (Open Web Text)

Max Activating Token Index: 452

Text #19

Max Range: 3.8720. Min Range: -3.8720

Max Act: 3.5303. Min Act: -0.1700

Data Index: 2767328 (Open Web Text)

Max Activating Token Index: 688