Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2252 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.6780. Min Act: -0.1700

Data Index: 5581127 (Open Web Text)

Max Activating Token Index: 684

Text #1

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.6544. Min Act: -0.1700

Data Index: 6964105 (Open Web Text)

Max Activating Token Index: 410

Text #2

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.6491. Min Act: -0.1700

Data Index: 2195616 (Open Web Text)

Max Activating Token Index: 990

Text #3

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.6421. Min Act: -0.1700

Data Index: 3432081 (Open Web Text)

Max Activating Token Index: 653

Text #4

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.6081. Min Act: -0.1700

Data Index: 4400611 (Open Web Text)

Max Activating Token Index: 125

Text #5

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.5325. Min Act: -0.1700

Data Index: 5346716 (Open Web Text)

Max Activating Token Index: 858

Text #6

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.5013. Min Act: -0.1700

Data Index: 4046783 (Open Web Text)

Max Activating Token Index: 868

Text #7

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4903. Min Act: -0.1700

Data Index: 5857476 (Open Web Text)

Max Activating Token Index: 650

Text #8

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4968. Min Act: -0.1700

Data Index: 1904788 (Open Web Text)

Max Activating Token Index: 284

Text #9

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4917. Min Act: -0.1700

Data Index: 5375038 (Open Web Text)

Max Activating Token Index: 396

Text #10

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4901. Min Act: -0.1700

Data Index: 6749916 (Open Web Text)

Max Activating Token Index: 557

Text #11

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4778. Min Act: -0.1700

Data Index: 1739235 (Open Web Text)

Max Activating Token Index: 846

Text #12

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4413. Min Act: -0.1700

Data Index: 6725283 (Open Web Text)

Max Activating Token Index: 844

Text #13

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4699. Min Act: -0.1700

Data Index: 1146876 (Open Web Text)

Max Activating Token Index: 254

Text #14

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4868. Min Act: -0.1700

Data Index: 1871332 (Open Web Text)

Max Activating Token Index: 801

Text #15

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4566. Min Act: -0.1700

Data Index: 4769547 (Open Web Text)

Max Activating Token Index: 244

Text #16

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4717. Min Act: -0.1700

Data Index: 2493931 (Open Web Text)

Max Activating Token Index: 38

Text #17

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4804. Min Act: -0.1700

Data Index: 1910506 (Open Web Text)

Max Activating Token Index: 436

Text #18

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4630. Min Act: -0.1700

Data Index: 5640066 (Open Web Text)

Max Activating Token Index: 990

Text #19

Max Range: 3.6780. Min Range: -3.6780

Max Act: 3.4451. Min Act: -0.1700

Data Index: 4520225 (Open Web Text)

Max Activating Token Index: 1011