Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1711 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1285. Min Range: -4.1285

Max Act: 4.1285. Min Act: -0.1700

Data Index: 4355046 (Open Web Text)

Max Activating Token Index: 301

Text #1

Max Range: 4.1285. Min Range: -4.1285

Max Act: 4.0832. Min Act: -0.1700

Data Index: 5639019 (Open Web Text)

Max Activating Token Index: 581

Text #2

Max Range: 4.1285. Min Range: -4.1285

Max Act: 4.0131. Min Act: -0.1700

Data Index: 8281634 (Open Web Text)

Max Activating Token Index: 606

Text #3

Max Range: 4.1285. Min Range: -4.1285

Max Act: 4.0093. Min Act: -0.1700

Data Index: 3064661 (Open Web Text)

Max Activating Token Index: 602

Text #4

Max Range: 4.1285. Min Range: -4.1285

Max Act: 4.0002. Min Act: -0.1700

Data Index: 6692041 (Open Web Text)

Max Activating Token Index: 832

Text #5

Max Range: 4.1285. Min Range: -4.1285

Max Act: 4.0144. Min Act: -0.1700

Data Index: 6832838 (Open Web Text)

Max Activating Token Index: 515

Text #6

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9826. Min Act: -0.1700

Data Index: 498664 (Open Web Text)

Max Activating Token Index: 855

Text #7

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9793. Min Act: -0.1700

Data Index: 95951 (Open Web Text)

Max Activating Token Index: 982

Text #8

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9833. Min Act: -0.1700

Data Index: 886853 (Open Web Text)

Max Activating Token Index: 1016

Text #9

Max Range: 4.1285. Min Range: -4.1285

Max Act: 4.0070. Min Act: -0.1700

Data Index: 5240670 (Open Web Text)

Max Activating Token Index: 83

Text #10

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9906. Min Act: -0.1700

Data Index: 2913633 (Open Web Text)

Max Activating Token Index: 850

Text #11

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9646. Min Act: -0.1700

Data Index: 6109545 (Open Web Text)

Max Activating Token Index: 993

Text #12

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9465. Min Act: -0.1700

Data Index: 2056408 (Open Web Text)

Max Activating Token Index: 28

Text #13

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9646. Min Act: -0.1700

Data Index: 1741738 (Open Web Text)

Max Activating Token Index: 866

Text #14

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9573. Min Act: -0.1700

Data Index: 445815 (Open Web Text)

Max Activating Token Index: 961

Text #15

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9687. Min Act: -0.1700

Data Index: 7101751 (Open Web Text)

Max Activating Token Index: 349

Text #16

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9673. Min Act: -0.1700

Data Index: 5199548 (Open Web Text)

Max Activating Token Index: 543

Text #17

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9498. Min Act: -0.1700

Data Index: 6535842 (Open Web Text)

Max Activating Token Index: 255

Text #18

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9347. Min Act: -0.1700

Data Index: 995732 (Open Web Text)

Max Activating Token Index: 229

Text #19

Max Range: 4.1285. Min Range: -4.1285

Max Act: 3.9204. Min Act: -0.1700

Data Index: 6003690 (Open Web Text)

Max Activating Token Index: 740