Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1239 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3791. Min Act: -0.1700

Data Index: 1319098 (Open Web Text)

Max Activating Token Index: 9

Text #1

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3843. Min Act: -0.1700

Data Index: 3003467 (Open Web Text)

Max Activating Token Index: 6

Text #2

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3916. Min Act: -0.1700

Data Index: 1734557 (Open Web Text)

Max Activating Token Index: 11

Text #3

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3974. Min Act: -0.1700

Data Index: 2197575 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3514. Min Act: -0.1700

Data Index: 7125586 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3629. Min Act: -0.1700

Data Index: 3864736 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3484. Min Act: -0.1700

Data Index: 746121 (Open Web Text)

Max Activating Token Index: 10

Text #7

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3362. Min Act: -0.1700

Data Index: 7323926 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3448. Min Act: -0.1700

Data Index: 8472581 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3230. Min Act: -0.1700

Data Index: 2752111 (Open Web Text)

Max Activating Token Index: 7

Text #10

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3128. Min Act: -0.1700

Data Index: 8641409 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3378. Min Act: -0.1700

Data Index: 4475032 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3211. Min Act: -0.1700

Data Index: 8687258 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3122. Min Act: -0.1700

Data Index: 7681675 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3101. Min Act: -0.1700

Data Index: 19194 (Open Web Text)

Max Activating Token Index: 11

Text #15

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.2866. Min Act: -0.1700

Data Index: 7797089 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.2805. Min Act: -0.1700

Data Index: 8063858 (Open Web Text)

Max Activating Token Index: 10

Text #17

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.2838. Min Act: -0.1700

Data Index: 3996398 (Open Web Text)

Max Activating Token Index: 5

Text #18

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.3044. Min Act: -0.1700

Data Index: 8258134 (Open Web Text)

Max Activating Token Index: 10

Text #19

Max Range: 4.3974. Min Range: -4.3974

Max Act: 4.2887. Min Act: -0.1700

Data Index: 1090770 (Open Web Text)

Max Activating Token Index: 6