Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 838 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.9620. Min Act: -0.1700

Data Index: 3898448 (Open Web Text)

Max Activating Token Index: 182

Text #1

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.9392. Min Act: -0.1700

Data Index: 715053 (Open Web Text)

Max Activating Token Index: 165

Text #2

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.9310. Min Act: -0.1700

Data Index: 7228318 (Open Web Text)

Max Activating Token Index: 131

Text #3

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.8960. Min Act: -0.1700

Data Index: 1399315 (Open Web Text)

Max Activating Token Index: 737

Text #4

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.8709. Min Act: -0.1700

Data Index: 5731861 (Open Web Text)

Max Activating Token Index: 878

Text #5

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.8935. Min Act: -0.1700

Data Index: 204557 (Open Web Text)

Max Activating Token Index: 865

Text #6

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.9065. Min Act: -0.1700

Data Index: 8797790 (Open Web Text)

Max Activating Token Index: 701

Text #7

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.8772. Min Act: -0.1700

Data Index: 7620257 (Open Web Text)

Max Activating Token Index: 46

Text #8

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.8427. Min Act: -0.1700

Data Index: 7703234 (Open Web Text)

Max Activating Token Index: 689

Text #9

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.8342. Min Act: -0.1700

Data Index: 3311457 (Open Web Text)

Max Activating Token Index: 721

Text #10

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.8486. Min Act: -0.1700

Data Index: 1468498 (Open Web Text)

Max Activating Token Index: 879

Text #11

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.8089. Min Act: -0.1700

Data Index: 3193533 (Open Web Text)

Max Activating Token Index: 520

Text #12

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.8108. Min Act: -0.1700

Data Index: 461717 (Open Web Text)

Max Activating Token Index: 37

Text #13

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.8387. Min Act: -0.1700

Data Index: 3670611 (Open Web Text)

Max Activating Token Index: 556

Text #14

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.8215. Min Act: -0.1700

Data Index: 7329415 (Open Web Text)

Max Activating Token Index: 664

Text #15

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.7865. Min Act: -0.1700

Data Index: 8442205 (Open Web Text)

Max Activating Token Index: 517

Text #16

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.7870. Min Act: -0.1700

Data Index: 6493403 (Open Web Text)

Max Activating Token Index: 837

Text #17

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.7618. Min Act: -0.1700

Data Index: 3459019 (Open Web Text)

Max Activating Token Index: 731

Text #18

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.7945. Min Act: -0.1700

Data Index: 1303064 (Open Web Text)

Max Activating Token Index: 351

Text #19

Max Range: 3.9620. Min Range: -3.9620

Max Act: 3.8005. Min Act: -0.1700

Data Index: 4831178 (Open Web Text)

Max Activating Token Index: 629