Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1222 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.9046. Min Act: -0.1700

Data Index: 1941081 (Open Web Text)

Max Activating Token Index: 318

Text #1

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.9121. Min Act: -0.1700

Data Index: 4742725 (Open Web Text)

Max Activating Token Index: 499

Text #2

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.8679. Min Act: -0.1700

Data Index: 6560567 (Open Web Text)

Max Activating Token Index: 507

Text #3

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.8738. Min Act: -0.1700

Data Index: 5965665 (Open Web Text)

Max Activating Token Index: 628

Text #4

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.8350. Min Act: -0.1700

Data Index: 5460215 (Open Web Text)

Max Activating Token Index: 322

Text #5

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.8689. Min Act: -0.1700

Data Index: 1824265 (Open Web Text)

Max Activating Token Index: 583

Text #6

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.8591. Min Act: -0.1700

Data Index: 5469625 (Open Web Text)

Max Activating Token Index: 220

Text #7

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.8496. Min Act: -0.1700

Data Index: 5035223 (Open Web Text)

Max Activating Token Index: 113

Text #8

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.8151. Min Act: -0.1700

Data Index: 5469611 (Open Web Text)

Max Activating Token Index: 839

Text #9

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.7631. Min Act: -0.1700

Data Index: 8694313 (Open Web Text)

Max Activating Token Index: 164

Text #10

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.7436. Min Act: -0.1700

Data Index: 6800351 (Open Web Text)

Max Activating Token Index: 911

Text #11

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.7493. Min Act: -0.1700

Data Index: 5419088 (Open Web Text)

Max Activating Token Index: 133

Text #12

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.7167. Min Act: -0.1700

Data Index: 8310948 (Open Web Text)

Max Activating Token Index: 98

Text #13

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.6890. Min Act: -0.1700

Data Index: 4026471 (Open Web Text)

Max Activating Token Index: 54

Text #14

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.6409. Min Act: -0.1700

Data Index: 2587221 (Open Web Text)

Max Activating Token Index: 917

Text #15

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.6757. Min Act: -0.1700

Data Index: 7069755 (Open Web Text)

Max Activating Token Index: 162

Text #16

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.6652. Min Act: -0.1700

Data Index: 7517158 (Open Web Text)

Max Activating Token Index: 634

Text #17

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.6991. Min Act: -0.1700

Data Index: 2565387 (Open Web Text)

Max Activating Token Index: 197

Text #18

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.6951. Min Act: -0.1700

Data Index: 6501792 (Open Web Text)

Max Activating Token Index: 601

Text #19

Max Range: 3.9121. Min Range: -3.9121

Max Act: 3.6746. Min Act: -0.1700

Data Index: 6690718 (Open Web Text)

Max Activating Token Index: 831