Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 32 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0928. Min Range: -4.0928

Max Act: 4.0892. Min Act: -0.1700

Data Index: 4562575 (Open Web Text)

Max Activating Token Index: 52

Text #1

Max Range: 4.0928. Min Range: -4.0928

Max Act: 4.0928. Min Act: -0.1700

Data Index: 4562574 (Open Web Text)

Max Activating Token Index: 485

Text #2

Max Range: 4.0928. Min Range: -4.0928

Max Act: 4.0538. Min Act: -0.1700

Data Index: 5395298 (Open Web Text)

Max Activating Token Index: 693

Text #3

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.9719. Min Act: -0.1700

Data Index: 2818066 (Open Web Text)

Max Activating Token Index: 973

Text #4

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.9324. Min Act: -0.1700

Data Index: 6421154 (Open Web Text)

Max Activating Token Index: 942

Text #5

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.8889. Min Act: -0.1700

Data Index: 5342124 (Open Web Text)

Max Activating Token Index: 726

Text #6

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.8499. Min Act: -0.1700

Data Index: 8184596 (Open Web Text)

Max Activating Token Index: 354

Text #7

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.8626. Min Act: -0.1700

Data Index: 623322 (Open Web Text)

Max Activating Token Index: 452

Text #8

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.8366. Min Act: -0.1700

Data Index: 5007724 (Open Web Text)

Max Activating Token Index: 655

Text #9

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.8520. Min Act: -0.1700

Data Index: 535405 (Open Web Text)

Max Activating Token Index: 218

Text #10

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.8440. Min Act: -0.1700

Data Index: 1059647 (Open Web Text)

Max Activating Token Index: 722

Text #11

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.8420. Min Act: -0.1700

Data Index: 6341537 (Open Web Text)

Max Activating Token Index: 552

Text #12

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.8374. Min Act: -0.1700

Data Index: 2395383 (Open Web Text)

Max Activating Token Index: 882

Text #13

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.8200. Min Act: -0.1700

Data Index: 3805011 (Open Web Text)

Max Activating Token Index: 739

Text #14

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.8052. Min Act: -0.1700

Data Index: 2040667 (Open Web Text)

Max Activating Token Index: 596

Text #15

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.8261. Min Act: -0.1700

Data Index: 2986436 (Open Web Text)

Max Activating Token Index: 326

Text #16

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.7914. Min Act: -0.1700

Data Index: 1673580 (Open Web Text)

Max Activating Token Index: 679

Text #17

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.7859. Min Act: -0.1700

Data Index: 1444213 (Open Web Text)

Max Activating Token Index: 352

Text #18

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.7987. Min Act: -0.1700

Data Index: 2294655 (Open Web Text)

Max Activating Token Index: 48

Text #19

Max Range: 4.0928. Min Range: -4.0928

Max Act: 3.7618. Min Act: -0.1700

Data Index: 5467070 (Open Web Text)

Max Activating Token Index: 811