Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 39 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.7988. Min Act: -0.1700

Data Index: 7904676 (Open Web Text)

Max Activating Token Index: 6

Text #1

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.7608. Min Act: -0.1700

Data Index: 4292157 (Open Web Text)

Max Activating Token Index: 2

Text #2

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.7608. Min Act: -0.1700

Data Index: 2550779 (Open Web Text)

Max Activating Token Index: 2

Text #3

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.7624. Min Act: -0.1700

Data Index: 948578 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.7423. Min Act: -0.1700

Data Index: 8158481 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.7252. Min Act: -0.1700

Data Index: 3698465 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.7272. Min Act: -0.1700

Data Index: 4052043 (Open Web Text)

Max Activating Token Index: 2

Text #7

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.7394. Min Act: -0.1700

Data Index: 3424635 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.7344. Min Act: -0.1700

Data Index: 6370999 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.7270. Min Act: -0.1700

Data Index: 6306843 (Open Web Text)

Max Activating Token Index: 2

Text #10

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.6873. Min Act: -0.1700

Data Index: 2621813 (Open Web Text)

Max Activating Token Index: 8

Text #11

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.6873. Min Act: -0.1700

Data Index: 6463977 (Open Web Text)

Max Activating Token Index: 4

Text #12

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.7050. Min Act: -0.1700

Data Index: 2189131 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.6939. Min Act: -0.1700

Data Index: 5035629 (Open Web Text)

Max Activating Token Index: 16

Text #14

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.6858. Min Act: -0.1700

Data Index: 4471854 (Open Web Text)

Max Activating Token Index: 3

Text #15

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.6975. Min Act: -0.1700

Data Index: 8690355 (Open Web Text)

Max Activating Token Index: 6

Text #16

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.6572. Min Act: -0.1700

Data Index: 5043654 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.6744. Min Act: -0.1700

Data Index: 5920204 (Open Web Text)

Max Activating Token Index: 3

Text #18

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.6638. Min Act: -0.1700

Data Index: 4447781 (Open Web Text)

Max Activating Token Index: 3

Text #19

Max Range: 4.7988. Min Range: -4.7988

Max Act: 4.6517. Min Act: -0.1700

Data Index: 6382637 (Open Web Text)

Max Activating Token Index: 5