Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1577 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.3280. Min Act: -0.1700

Data Index: 3197673 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.3085. Min Act: -0.1700

Data Index: 2441978 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.3049. Min Act: -0.1700

Data Index: 5174311 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.2698. Min Act: -0.1700

Data Index: 1529827 (Open Web Text)

Max Activating Token Index: 8

Text #4

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.2606. Min Act: -0.1700

Data Index: 402711 (Open Web Text)

Max Activating Token Index: 7

Text #5

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.2606. Min Act: -0.1700

Data Index: 402712 (Open Web Text)

Max Activating Token Index: 7

Text #6

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.2358. Min Act: -0.1700

Data Index: 7100225 (Open Web Text)

Max Activating Token Index: 2

Text #7

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.2358. Min Act: -0.1700

Data Index: 456399 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.2300. Min Act: -0.1700

Data Index: 402714 (Open Web Text)

Max Activating Token Index: 8

Text #9

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.2358. Min Act: -0.1700

Data Index: 8827322 (Open Web Text)

Max Activating Token Index: 2

Text #10

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.2234. Min Act: -0.1700

Data Index: 2207443 (Open Web Text)

Max Activating Token Index: 7

Text #11

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.2121. Min Act: -0.1700

Data Index: 402717 (Open Web Text)

Max Activating Token Index: 10

Text #12

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.2108. Min Act: -0.1700

Data Index: 6921522 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.2110. Min Act: -0.1700

Data Index: 8383551 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.1925. Min Act: -0.1700

Data Index: 8233829 (Open Web Text)

Max Activating Token Index: 8

Text #15

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.1862. Min Act: -0.1700

Data Index: 2364208 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.1810. Min Act: -0.1700

Data Index: 3172520 (Open Web Text)

Max Activating Token Index: 14

Text #17

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.1748. Min Act: -0.1700

Data Index: 3733 (Open Web Text)

Max Activating Token Index: 3

Text #18

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.1615. Min Act: -0.1700

Data Index: 402713 (Open Web Text)

Max Activating Token Index: 19

Text #19

Max Range: 3.3280. Min Range: -3.3280

Max Act: 3.1545. Min Act: -0.1700

Data Index: 5807434 (Open Web Text)

Max Activating Token Index: 3