Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2827 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.5447. Min Act: -0.1695

Data Index: 3812735 (Open Web Text)

Max Activating Token Index: 2

Text #1

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.5549. Min Act: -0.1700

Data Index: 7998300 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.5248. Min Act: -0.1700

Data Index: 1054560 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.5279. Min Act: -0.1700

Data Index: 6685552 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.4916. Min Act: -0.1678

Data Index: 7664229 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.5248. Min Act: -0.1700

Data Index: 96653 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.4729. Min Act: -0.1700

Data Index: 5987940 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.4480. Min Act: -0.1700

Data Index: 3352663 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.4357. Min Act: -0.1697

Data Index: 3098326 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.4606. Min Act: -0.1700

Data Index: 8631780 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.4608. Min Act: -0.1699

Data Index: 1815497 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.4395. Min Act: -0.1689

Data Index: 434773 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.4415. Min Act: -0.1652

Data Index: 6308177 (Open Web Text)

Max Activating Token Index: 2

Text #13

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.4717. Min Act: -0.1700

Data Index: 1212095 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.3892. Min Act: -0.1700

Data Index: 875756 (Open Web Text)

Max Activating Token Index: 3

Text #15

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.3789. Min Act: -0.1700

Data Index: 3445302 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.3974. Min Act: -0.1700

Data Index: 3544675 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.4291. Min Act: -0.1700

Data Index: 285086 (Open Web Text)

Max Activating Token Index: 8

Text #18

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.3926. Min Act: -0.1700

Data Index: 4116862 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 7.5549. Min Range: -7.5549

Max Act: 7.4194. Min Act: -0.1700

Data Index: 4433060 (Open Web Text)

Max Activating Token Index: 3