Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 936 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.8640. Min Act: -0.1700

Data Index: 6010902 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.8331. Min Act: -0.1700

Data Index: 1283949 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.8286. Min Act: -0.1700

Data Index: 5480214 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.8248. Min Act: -0.1700

Data Index: 6515348 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.8181. Min Act: -0.1700

Data Index: 5632131 (Open Web Text)

Max Activating Token Index: 9

Text #5

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7976. Min Act: -0.1700

Data Index: 5323348 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.8016. Min Act: -0.1700

Data Index: 8073369 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7973. Min Act: -0.1700

Data Index: 7827453 (Open Web Text)

Max Activating Token Index: 18

Text #8

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7797. Min Act: -0.1700

Data Index: 234459 (Open Web Text)

Max Activating Token Index: 7

Text #9

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7896. Min Act: -0.1700

Data Index: 7277518 (Open Web Text)

Max Activating Token Index: 4

Text #10

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7926. Min Act: -0.1700

Data Index: 3575840 (Open Web Text)

Max Activating Token Index: 2

Text #11

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7926. Min Act: -0.1700

Data Index: 3751926 (Open Web Text)

Max Activating Token Index: 2

Text #12

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7766. Min Act: -0.1700

Data Index: 7832860 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7944. Min Act: -0.1700

Data Index: 4989479 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7782. Min Act: -0.1700

Data Index: 8343255 (Open Web Text)

Max Activating Token Index: 8

Text #15

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7745. Min Act: -0.1700

Data Index: 5905831 (Open Web Text)

Max Activating Token Index: 5

Text #16

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7638. Min Act: -0.1700

Data Index: 5696211 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7772. Min Act: -0.1700

Data Index: 463922 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7722. Min Act: -0.1700

Data Index: 3941138 (Open Web Text)

Max Activating Token Index: 21

Text #19

Max Range: 2.8640. Min Range: -2.8640

Max Act: 2.7639. Min Act: -0.1700

Data Index: 4169266 (Open Web Text)

Max Activating Token Index: 4