Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1129 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.6120. Min Act: -0.1699

Data Index: 5875838 (Open Web Text)

Max Activating Token Index: 235

Text #1

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.6024. Min Act: -0.1695

Data Index: 7649717 (Open Web Text)

Max Activating Token Index: 190

Text #2

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.6006. Min Act: -0.1697

Data Index: 6893080 (Open Web Text)

Max Activating Token Index: 869

Text #3

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.6085. Min Act: -0.1681

Data Index: 6010811 (Open Web Text)

Max Activating Token Index: 171

Text #4

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.6003. Min Act: -0.1700

Data Index: 420852 (Open Web Text)

Max Activating Token Index: 55

Text #5

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5923. Min Act: -0.1700

Data Index: 3379462 (Open Web Text)

Max Activating Token Index: 440

Text #6

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5883. Min Act: -0.1700

Data Index: 4723716 (Open Web Text)

Max Activating Token Index: 58

Text #7

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5917. Min Act: -0.1700

Data Index: 1609268 (Open Web Text)

Max Activating Token Index: 146

Text #8

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5702. Min Act: -0.1697

Data Index: 2096597 (Open Web Text)

Max Activating Token Index: 283

Text #9

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5731. Min Act: -0.1696

Data Index: 7481241 (Open Web Text)

Max Activating Token Index: 35

Text #10

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5785. Min Act: -0.1700

Data Index: 1452359 (Open Web Text)

Max Activating Token Index: 167

Text #11

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5694. Min Act: -0.1675

Data Index: 6404364 (Open Web Text)

Max Activating Token Index: 545

Text #12

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5686. Min Act: -0.1700

Data Index: 416625 (Open Web Text)

Max Activating Token Index: 44

Text #13

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5676. Min Act: -0.1700

Data Index: 6760789 (Open Web Text)

Max Activating Token Index: 25

Text #14

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5557. Min Act: -0.1694

Data Index: 3594173 (Open Web Text)

Max Activating Token Index: 238

Text #15

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5745. Min Act: -0.1700

Data Index: 618888 (Open Web Text)

Max Activating Token Index: 276

Text #16

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5601. Min Act: -0.1688

Data Index: 3275726 (Open Web Text)

Max Activating Token Index: 105

Text #17

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5499. Min Act: -0.1698

Data Index: 2845940 (Open Web Text)

Max Activating Token Index: 545

Text #18

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5582. Min Act: -0.1697

Data Index: 3322447 (Open Web Text)

Max Activating Token Index: 596

Text #19

Max Range: 1.6120. Min Range: -1.6120

Max Act: 1.5579. Min Act: -0.1700

Data Index: 3580166 (Open Web Text)

Max Activating Token Index: 42