Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 681 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0146. Min Range: -4.0146

Max Act: 4.0146. Min Act: -0.1700

Data Index: 1101301 (Open Web Text)

Max Activating Token Index: 89

Text #1

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.9771. Min Act: -0.1700

Data Index: 688464 (Open Web Text)

Max Activating Token Index: 811

Text #2

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.8925. Min Act: -0.1700

Data Index: 688465 (Open Web Text)

Max Activating Token Index: 816

Text #3

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.8434. Min Act: -0.1700

Data Index: 5447761 (Open Web Text)

Max Activating Token Index: 643

Text #4

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.8475. Min Act: -0.1700

Data Index: 6579566 (Open Web Text)

Max Activating Token Index: 149

Text #5

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.8146. Min Act: -0.1700

Data Index: 4610511 (Open Web Text)

Max Activating Token Index: 913

Text #6

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7884. Min Act: -0.1700

Data Index: 4255845 (Open Web Text)

Max Activating Token Index: 197

Text #7

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7900. Min Act: -0.1700

Data Index: 4414023 (Open Web Text)

Max Activating Token Index: 118

Text #8

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7915. Min Act: -0.1700

Data Index: 8497721 (Open Web Text)

Max Activating Token Index: 549

Text #9

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7912. Min Act: -0.1700

Data Index: 8676501 (Open Web Text)

Max Activating Token Index: 652

Text #10

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7655. Min Act: -0.1700

Data Index: 1223007 (Open Web Text)

Max Activating Token Index: 268

Text #11

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7561. Min Act: -0.1700

Data Index: 4800536 (Open Web Text)

Max Activating Token Index: 893

Text #12

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7743. Min Act: -0.1700

Data Index: 4621208 (Open Web Text)

Max Activating Token Index: 113

Text #13

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7507. Min Act: -0.1700

Data Index: 6686063 (Open Web Text)

Max Activating Token Index: 563

Text #14

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7260. Min Act: -0.1700

Data Index: 4621207 (Open Web Text)

Max Activating Token Index: 866

Text #15

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7447. Min Act: -0.1700

Data Index: 5832843 (Open Web Text)

Max Activating Token Index: 626

Text #16

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7153. Min Act: -0.1700

Data Index: 3185187 (Open Web Text)

Max Activating Token Index: 325

Text #17

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7349. Min Act: -0.1700

Data Index: 7868330 (Open Web Text)

Max Activating Token Index: 379

Text #18

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7320. Min Act: -0.1700

Data Index: 5217425 (Open Web Text)

Max Activating Token Index: 568

Text #19

Max Range: 4.0146. Min Range: -4.0146

Max Act: 3.7155. Min Act: -0.1700

Data Index: 6092282 (Open Web Text)

Max Activating Token Index: 391