Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 997 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0390. Min Range: -4.0390

Max Act: 4.0370. Min Act: -0.1700

Data Index: 8424587 (Open Web Text)

Max Activating Token Index: 326

Text #1

Max Range: 4.0390. Min Range: -4.0390

Max Act: 4.0390. Min Act: -0.1700

Data Index: 5065130 (Open Web Text)

Max Activating Token Index: 493

Text #2

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.9544. Min Act: -0.1700

Data Index: 5214669 (Open Web Text)

Max Activating Token Index: 151

Text #3

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.9217. Min Act: -0.1700

Data Index: 5431593 (Open Web Text)

Max Activating Token Index: 523

Text #4

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.9120. Min Act: -0.1700

Data Index: 4338479 (Open Web Text)

Max Activating Token Index: 985

Text #5

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.9276. Min Act: -0.1700

Data Index: 4433589 (Open Web Text)

Max Activating Token Index: 412

Text #6

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.8732. Min Act: -0.1700

Data Index: 3004786 (Open Web Text)

Max Activating Token Index: 845

Text #7

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.8781. Min Act: -0.1700

Data Index: 3281585 (Open Web Text)

Max Activating Token Index: 916

Text #8

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.9068. Min Act: -0.1700

Data Index: 6148163 (Open Web Text)

Max Activating Token Index: 442

Text #9

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.9023. Min Act: -0.1700

Data Index: 5692025 (Open Web Text)

Max Activating Token Index: 944

Text #10

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.9131. Min Act: -0.1700

Data Index: 7557002 (Open Web Text)

Max Activating Token Index: 1010

Text #11

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.9101. Min Act: -0.1700

Data Index: 2917381 (Open Web Text)

Max Activating Token Index: 318

Text #12

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.9037. Min Act: -0.1700

Data Index: 4056299 (Open Web Text)

Max Activating Token Index: 409

Text #13

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.8960. Min Act: -0.1700

Data Index: 3748766 (Open Web Text)

Max Activating Token Index: 219

Text #14

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.8932. Min Act: -0.1700

Data Index: 5606271 (Open Web Text)

Max Activating Token Index: 967

Text #15

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.8947. Min Act: -0.1700

Data Index: 729319 (Open Web Text)

Max Activating Token Index: 843

Text #16

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.8822. Min Act: -0.1700

Data Index: 1731727 (Open Web Text)

Max Activating Token Index: 529

Text #17

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.8501. Min Act: -0.1700

Data Index: 5605463 (Open Web Text)

Max Activating Token Index: 717

Text #18

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.8570. Min Act: -0.1700

Data Index: 992877 (Open Web Text)

Max Activating Token Index: 533

Text #19

Max Range: 4.0390. Min Range: -4.0390

Max Act: 3.8651. Min Act: -0.1700

Data Index: 3374674 (Open Web Text)

Max Activating Token Index: 108