Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 139 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0668. Min Range: -3.0668

Max Act: 3.0668. Min Act: -0.1700

Data Index: 5816934 (Open Web Text)

Max Activating Token Index: 269

Text #1

Max Range: 3.0668. Min Range: -3.0668

Max Act: 3.0147. Min Act: -0.1700

Data Index: 6227938 (Open Web Text)

Max Activating Token Index: 647

Text #2

Max Range: 3.0668. Min Range: -3.0668

Max Act: 3.0027. Min Act: -0.1700

Data Index: 8812644 (Open Web Text)

Max Activating Token Index: 157

Text #3

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.9845. Min Act: -0.1700

Data Index: 2735270 (Open Web Text)

Max Activating Token Index: 1004

Text #4

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.9892. Min Act: -0.1700

Data Index: 4610568 (Open Web Text)

Max Activating Token Index: 403

Text #5

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.9508. Min Act: -0.1700

Data Index: 4356854 (Open Web Text)

Max Activating Token Index: 985

Text #6

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.9667. Min Act: -0.1700

Data Index: 6326040 (Open Web Text)

Max Activating Token Index: 679

Text #7

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.9290. Min Act: -0.1700

Data Index: 1623365 (Open Web Text)

Max Activating Token Index: 100

Text #8

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.9129. Min Act: -0.1700

Data Index: 6036733 (Open Web Text)

Max Activating Token Index: 515

Text #9

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.9089. Min Act: -0.1700

Data Index: 3561382 (Open Web Text)

Max Activating Token Index: 678

Text #10

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.8959. Min Act: -0.1700

Data Index: 5952722 (Open Web Text)

Max Activating Token Index: 117

Text #11

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.9123. Min Act: -0.1700

Data Index: 1866297 (Open Web Text)

Max Activating Token Index: 533

Text #12

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.9104. Min Act: -0.1700

Data Index: 6527493 (Open Web Text)

Max Activating Token Index: 781

Text #13

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.8881. Min Act: -0.1700

Data Index: 7857528 (Open Web Text)

Max Activating Token Index: 474

Text #14

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.8656. Min Act: -0.1700

Data Index: 3113387 (Open Web Text)

Max Activating Token Index: 601

Text #15

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.8714. Min Act: -0.1700

Data Index: 8743101 (Open Web Text)

Max Activating Token Index: 294

Text #16

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.8658. Min Act: -0.1700

Data Index: 4176588 (Open Web Text)

Max Activating Token Index: 863

Text #17

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.8424. Min Act: -0.1700

Data Index: 7633043 (Open Web Text)

Max Activating Token Index: 764

Text #18

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.8538. Min Act: -0.1700

Data Index: 7610616 (Open Web Text)

Max Activating Token Index: 696

Text #19

Max Range: 3.0668. Min Range: -3.0668

Max Act: 2.8770. Min Act: -0.1700

Data Index: 2008854 (Open Web Text)

Max Activating Token Index: 224