Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1207 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.7123. Min Act: -0.1700

Data Index: 4141528 (Open Web Text)

Max Activating Token Index: 37

Text #1

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.6904. Min Act: -0.1700

Data Index: 8364589 (Open Web Text)

Max Activating Token Index: 15

Text #2

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.6447. Min Act: -0.1700

Data Index: 6081504 (Open Web Text)

Max Activating Token Index: 524

Text #3

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.6304. Min Act: -0.1700

Data Index: 5388801 (Open Web Text)

Max Activating Token Index: 23

Text #4

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.6192. Min Act: -0.1700

Data Index: 1298053 (Open Web Text)

Max Activating Token Index: 181

Text #5

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5649. Min Act: -0.1700

Data Index: 6194727 (Open Web Text)

Max Activating Token Index: 547

Text #6

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5871. Min Act: -0.1700

Data Index: 755491 (Open Web Text)

Max Activating Token Index: 42

Text #7

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.6115. Min Act: -0.1700

Data Index: 2750325 (Open Web Text)

Max Activating Token Index: 345

Text #8

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5937. Min Act: -0.1700

Data Index: 3724411 (Open Web Text)

Max Activating Token Index: 37

Text #9

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5803. Min Act: -0.1700

Data Index: 2330658 (Open Web Text)

Max Activating Token Index: 765

Text #10

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.6075. Min Act: -0.1700

Data Index: 1291356 (Open Web Text)

Max Activating Token Index: 173

Text #11

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5738. Min Act: -0.1700

Data Index: 1574429 (Open Web Text)

Max Activating Token Index: 360

Text #12

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5823. Min Act: -0.1700

Data Index: 1329306 (Open Web Text)

Max Activating Token Index: 122

Text #13

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5852. Min Act: -0.1700

Data Index: 4641417 (Open Web Text)

Max Activating Token Index: 903

Text #14

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5724. Min Act: -0.1700

Data Index: 7022684 (Open Web Text)

Max Activating Token Index: 120

Text #15

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5197. Min Act: -0.1700

Data Index: 2398400 (Open Web Text)

Max Activating Token Index: 331

Text #16

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5405. Min Act: -0.1700

Data Index: 1711457 (Open Web Text)

Max Activating Token Index: 655

Text #17

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5288. Min Act: -0.1700

Data Index: 2601491 (Open Web Text)

Max Activating Token Index: 743

Text #18

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5480. Min Act: -0.1700

Data Index: 7011777 (Open Web Text)

Max Activating Token Index: 20

Text #19

Max Range: 4.7123. Min Range: -4.7123

Max Act: 4.5212. Min Act: -0.1700

Data Index: 4296814 (Open Web Text)

Max Activating Token Index: 381