Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1070 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.6290. Min Act: -0.1682

Data Index: 7463668 (Open Web Text)

Max Activating Token Index: 652

Text #1

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5900. Min Act: -0.1696

Data Index: 8830104 (Open Web Text)

Max Activating Token Index: 634

Text #2

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.6128. Min Act: -0.1557

Data Index: 7515507 (Open Web Text)

Max Activating Token Index: 1018

Text #3

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5864. Min Act: -0.1683

Data Index: 8785665 (Open Web Text)

Max Activating Token Index: 737

Text #4

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5832. Min Act: -0.1699

Data Index: 8335515 (Open Web Text)

Max Activating Token Index: 669

Text #5

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5841. Min Act: -0.1586

Data Index: 6118504 (Open Web Text)

Max Activating Token Index: 663

Text #6

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5970. Min Act: -0.1621

Data Index: 2822458 (Open Web Text)

Max Activating Token Index: 1021

Text #7

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5841. Min Act: -0.1557

Data Index: 5592214 (Open Web Text)

Max Activating Token Index: 615

Text #8

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5809. Min Act: -0.1613

Data Index: 2165307 (Open Web Text)

Max Activating Token Index: 712

Text #9

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5823. Min Act: -0.1557

Data Index: 2718325 (Open Web Text)

Max Activating Token Index: 664

Text #10

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5812. Min Act: -0.1557

Data Index: 723914 (Open Web Text)

Max Activating Token Index: 1022

Text #11

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5927. Min Act: -0.1638

Data Index: 3357763 (Open Web Text)

Max Activating Token Index: 118

Text #12

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5845. Min Act: -0.1606

Data Index: 2121669 (Open Web Text)

Max Activating Token Index: 661

Text #13

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5731. Min Act: -0.1573

Data Index: 319202 (Open Web Text)

Max Activating Token Index: 738

Text #14

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5676. Min Act: -0.1638

Data Index: 7553660 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5589. Min Act: -0.1699

Data Index: 4044966 (Open Web Text)

Max Activating Token Index: 1021

Text #16

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5708. Min Act: -0.1640

Data Index: 848666 (Open Web Text)

Max Activating Token Index: 580

Text #17

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5719. Min Act: -0.1634

Data Index: 4754806 (Open Web Text)

Max Activating Token Index: 602

Text #18

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5547. Min Act: -0.1636

Data Index: 1818457 (Open Web Text)

Max Activating Token Index: 570

Text #19

Max Range: 1.6290. Min Range: -1.6290

Max Act: 1.5647. Min Act: -0.1697

Data Index: 331982 (Open Web Text)

Max Activating Token Index: 651