Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 247 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.6722. Min Act: -0.1700

Data Index: 6071997 (Open Web Text)

Max Activating Token Index: 11

Text #1

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.6508. Min Act: -0.1700

Data Index: 8523031 (Open Web Text)

Max Activating Token Index: 84

Text #2

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.6258. Min Act: -0.1700

Data Index: 4179607 (Open Web Text)

Max Activating Token Index: 55

Text #3

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.6318. Min Act: -0.1700

Data Index: 6217600 (Open Web Text)

Max Activating Token Index: 35

Text #4

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.6204. Min Act: -0.1700

Data Index: 5580740 (Open Web Text)

Max Activating Token Index: 986

Text #5

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5944. Min Act: -0.1700

Data Index: 1959770 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5933. Min Act: -0.1700

Data Index: 7740583 (Open Web Text)

Max Activating Token Index: 556

Text #7

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5889. Min Act: -0.1700

Data Index: 301621 (Open Web Text)

Max Activating Token Index: 988

Text #8

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5937. Min Act: -0.1700

Data Index: 8049315 (Open Web Text)

Max Activating Token Index: 204

Text #9

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5788. Min Act: -0.1700

Data Index: 6712372 (Open Web Text)

Max Activating Token Index: 487

Text #10

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5880. Min Act: -0.1700

Data Index: 1469564 (Open Web Text)

Max Activating Token Index: 44

Text #11

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5771. Min Act: -0.1700

Data Index: 7326343 (Open Web Text)

Max Activating Token Index: 41

Text #12

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5638. Min Act: -0.1700

Data Index: 8031913 (Open Web Text)

Max Activating Token Index: 637

Text #13

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5640. Min Act: -0.1700

Data Index: 4948059 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5654. Min Act: -0.1700

Data Index: 8513897 (Open Web Text)

Max Activating Token Index: 683

Text #15

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5623. Min Act: -0.1700

Data Index: 7959603 (Open Web Text)

Max Activating Token Index: 286

Text #16

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5564. Min Act: -0.1700

Data Index: 5943980 (Open Web Text)

Max Activating Token Index: 169

Text #17

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5550. Min Act: -0.1700

Data Index: 4118000 (Open Web Text)

Max Activating Token Index: 91

Text #18

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5524. Min Act: -0.1700

Data Index: 5986725 (Open Web Text)

Max Activating Token Index: 771

Text #19

Max Range: 1.6722. Min Range: -1.6722

Max Act: 1.5638. Min Act: -0.1700

Data Index: 3810241 (Open Web Text)

Max Activating Token Index: 439