Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1098 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0696. Min Range: -4.0696

Max Act: 4.0696. Min Act: -0.1700

Data Index: 5115348 (Open Web Text)

Max Activating Token Index: 843

Text #1

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.9717. Min Act: -0.1700

Data Index: 8656857 (Open Web Text)

Max Activating Token Index: 980

Text #2

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.9223. Min Act: -0.1700

Data Index: 2835985 (Open Web Text)

Max Activating Token Index: 568

Text #3

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.9153. Min Act: -0.1700

Data Index: 3490621 (Open Web Text)

Max Activating Token Index: 219

Text #4

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.9133. Min Act: -0.1700

Data Index: 6555781 (Open Web Text)

Max Activating Token Index: 383

Text #5

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.8420. Min Act: -0.1700

Data Index: 129987 (Open Web Text)

Max Activating Token Index: 454

Text #6

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.8304. Min Act: -0.1700

Data Index: 1431268 (Open Web Text)

Max Activating Token Index: 136

Text #7

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.8173. Min Act: -0.1700

Data Index: 4842134 (Open Web Text)

Max Activating Token Index: 981

Text #8

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.8279. Min Act: -0.1700

Data Index: 6145647 (Open Web Text)

Max Activating Token Index: 570

Text #9

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.8085. Min Act: -0.1700

Data Index: 7790288 (Open Web Text)

Max Activating Token Index: 899

Text #10

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.7827. Min Act: -0.1700

Data Index: 8312070 (Open Web Text)

Max Activating Token Index: 931

Text #11

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.7971. Min Act: -0.1700

Data Index: 6698929 (Open Web Text)

Max Activating Token Index: 871

Text #12

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.7807. Min Act: -0.1700

Data Index: 393857 (Open Web Text)

Max Activating Token Index: 553

Text #13

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.7396. Min Act: -0.1700

Data Index: 3919864 (Open Web Text)

Max Activating Token Index: 127

Text #14

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.7422. Min Act: -0.1700

Data Index: 4725620 (Open Web Text)

Max Activating Token Index: 349

Text #15

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.7278. Min Act: -0.1700

Data Index: 4557505 (Open Web Text)

Max Activating Token Index: 931

Text #16

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.7501. Min Act: -0.1700

Data Index: 1209900 (Open Web Text)

Max Activating Token Index: 633

Text #17

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.7179. Min Act: -0.1700

Data Index: 2130992 (Open Web Text)

Max Activating Token Index: 806

Text #18

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.7190. Min Act: -0.1700

Data Index: 6950316 (Open Web Text)

Max Activating Token Index: 974

Text #19

Max Range: 4.0696. Min Range: -4.0696

Max Act: 3.7152. Min Act: -0.1700

Data Index: 435504 (Open Web Text)

Max Activating Token Index: 283