Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 553 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1130. Min Range: -5.1130

Max Act: 5.1130. Min Act: -0.1700

Data Index: 3470791 (Open Web Text)

Max Activating Token Index: 499

Text #1

Max Range: 5.1130. Min Range: -5.1130

Max Act: 5.0241. Min Act: -0.1700

Data Index: 6185145 (Open Web Text)

Max Activating Token Index: 732

Text #2

Max Range: 5.1130. Min Range: -5.1130

Max Act: 5.0090. Min Act: -0.1700

Data Index: 8501162 (Open Web Text)

Max Activating Token Index: 628

Text #3

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8739. Min Act: -0.1700

Data Index: 8397733 (Open Web Text)

Max Activating Token Index: 821

Text #4

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.9165. Min Act: -0.1700

Data Index: 5145233 (Open Web Text)

Max Activating Token Index: 411

Text #5

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8675. Min Act: -0.1700

Data Index: 5798704 (Open Web Text)

Max Activating Token Index: 990

Text #6

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8601. Min Act: -0.1700

Data Index: 373306 (Open Web Text)

Max Activating Token Index: 503

Text #7

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8995. Min Act: -0.1700

Data Index: 6931649 (Open Web Text)

Max Activating Token Index: 637

Text #8

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8894. Min Act: -0.1700

Data Index: 5705950 (Open Web Text)

Max Activating Token Index: 528

Text #9

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8986. Min Act: -0.1700

Data Index: 1119044 (Open Web Text)

Max Activating Token Index: 688

Text #10

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8828. Min Act: -0.1700

Data Index: 5174239 (Open Web Text)

Max Activating Token Index: 144

Text #11

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8154. Min Act: -0.1700

Data Index: 4575084 (Open Web Text)

Max Activating Token Index: 917

Text #12

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8373. Min Act: -0.1700

Data Index: 1541526 (Open Web Text)

Max Activating Token Index: 256

Text #13

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8108. Min Act: -0.1700

Data Index: 9673 (Open Web Text)

Max Activating Token Index: 458

Text #14

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8607. Min Act: -0.1700

Data Index: 6799726 (Open Web Text)

Max Activating Token Index: 688

Text #15

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8198. Min Act: -0.1700

Data Index: 8402302 (Open Web Text)

Max Activating Token Index: 133

Text #16

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8040. Min Act: -0.1700

Data Index: 580818 (Open Web Text)

Max Activating Token Index: 189

Text #17

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.7624. Min Act: -0.1700

Data Index: 3706892 (Open Web Text)

Max Activating Token Index: 565

Text #18

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.8252. Min Act: -0.1700

Data Index: 8241947 (Open Web Text)

Max Activating Token Index: 1020

Text #19

Max Range: 5.1130. Min Range: -5.1130

Max Act: 4.7473. Min Act: -0.1700

Data Index: 7032686 (Open Web Text)

Max Activating Token Index: 696