Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 196 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.3477. Min Act: -0.1700

Data Index: 687665 (Open Web Text)

Max Activating Token Index: 234

Text #1

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.3721. Min Act: -0.1700

Data Index: 8441886 (Open Web Text)

Max Activating Token Index: 707

Text #2

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.2977. Min Act: -0.1700

Data Index: 2738082 (Open Web Text)

Max Activating Token Index: 11

Text #3

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.3531. Min Act: -0.1700

Data Index: 6562645 (Open Web Text)

Max Activating Token Index: 403

Text #4

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.2729. Min Act: -0.1700

Data Index: 5341818 (Open Web Text)

Max Activating Token Index: 515

Text #5

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.2915. Min Act: -0.1700

Data Index: 2587691 (Open Web Text)

Max Activating Token Index: 159

Text #6

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.2877. Min Act: -0.1700

Data Index: 5524855 (Open Web Text)

Max Activating Token Index: 47

Text #7

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.2214. Min Act: -0.1700

Data Index: 3658813 (Open Web Text)

Max Activating Token Index: 29

Text #8

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.2544. Min Act: -0.1700

Data Index: 1193371 (Open Web Text)

Max Activating Token Index: 394

Text #9

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.2310. Min Act: -0.1700

Data Index: 3734649 (Open Web Text)

Max Activating Token Index: 684

Text #10

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.2270. Min Act: -0.1700

Data Index: 383563 (Open Web Text)

Max Activating Token Index: 40

Text #11

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.2316. Min Act: -0.1700

Data Index: 2960009 (Open Web Text)

Max Activating Token Index: 39

Text #12

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.2025. Min Act: -0.1700

Data Index: 6436699 (Open Web Text)

Max Activating Token Index: 502

Text #13

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.1887. Min Act: -0.1700

Data Index: 4831409 (Open Web Text)

Max Activating Token Index: 23

Text #14

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.1802. Min Act: -0.1700

Data Index: 5779118 (Open Web Text)

Max Activating Token Index: 13

Text #15

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.1490. Min Act: -0.1700

Data Index: 7850914 (Open Web Text)

Max Activating Token Index: 458

Text #16

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.1845. Min Act: -0.1700

Data Index: 1023602 (Open Web Text)

Max Activating Token Index: 33

Text #17

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.1467. Min Act: -0.1700

Data Index: 5003837 (Open Web Text)

Max Activating Token Index: 375

Text #18

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.1281. Min Act: -0.1700

Data Index: 8423366 (Open Web Text)

Max Activating Token Index: 608

Text #19

Max Range: 4.3721. Min Range: -4.3721

Max Act: 4.1275. Min Act: -0.1700

Data Index: 519166 (Open Web Text)

Max Activating Token Index: 579