Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1786 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.7611. Min Act: -0.1700

Data Index: 7827016 (Open Web Text)

Max Activating Token Index: 67

Text #1

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.6916. Min Act: -0.1700

Data Index: 6895320 (Open Web Text)

Max Activating Token Index: 192

Text #2

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.6632. Min Act: -0.1700

Data Index: 4471637 (Open Web Text)

Max Activating Token Index: 35

Text #3

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.5907. Min Act: -0.1700

Data Index: 5675094 (Open Web Text)

Max Activating Token Index: 148

Text #4

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.5319. Min Act: -0.1700

Data Index: 942068 (Open Web Text)

Max Activating Token Index: 931

Text #5

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.5317. Min Act: -0.1700

Data Index: 7650568 (Open Web Text)

Max Activating Token Index: 877

Text #6

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.5211. Min Act: -0.1700

Data Index: 4563015 (Open Web Text)

Max Activating Token Index: 39

Text #7

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.5225. Min Act: -0.1700

Data Index: 6217372 (Open Web Text)

Max Activating Token Index: 85

Text #8

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.5149. Min Act: -0.1700

Data Index: 7734649 (Open Web Text)

Max Activating Token Index: 52

Text #9

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.4785. Min Act: -0.1700

Data Index: 5938234 (Open Web Text)

Max Activating Token Index: 338

Text #10

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.4549. Min Act: -0.1700

Data Index: 1957962 (Open Web Text)

Max Activating Token Index: 63

Text #11

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.4497. Min Act: -0.1700

Data Index: 7118538 (Open Web Text)

Max Activating Token Index: 232

Text #12

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.4480. Min Act: -0.1700

Data Index: 8169838 (Open Web Text)

Max Activating Token Index: 247

Text #13

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.4362. Min Act: -0.1700

Data Index: 5455496 (Open Web Text)

Max Activating Token Index: 44

Text #14

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.4448. Min Act: -0.1700

Data Index: 7766368 (Open Web Text)

Max Activating Token Index: 38

Text #15

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.4657. Min Act: -0.1700

Data Index: 5429039 (Open Web Text)

Max Activating Token Index: 464

Text #16

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.4188. Min Act: -0.1700

Data Index: 4673968 (Open Web Text)

Max Activating Token Index: 697

Text #17

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.4212. Min Act: -0.1700

Data Index: 6231395 (Open Web Text)

Max Activating Token Index: 578

Text #18

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.4123. Min Act: -0.1700

Data Index: 1810991 (Open Web Text)

Max Activating Token Index: 530

Text #19

Max Range: 3.7611. Min Range: -3.7611

Max Act: 3.4191. Min Act: -0.1700

Data Index: 3967091 (Open Web Text)

Max Activating Token Index: 483