Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 29 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.8860. Min Act: -0.1700

Data Index: 5708964 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.8822. Min Act: -0.1700

Data Index: 264585 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.8549. Min Act: -0.1700

Data Index: 4437535 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.8481. Min Act: -0.1700

Data Index: 4288045 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.8336. Min Act: -0.1700

Data Index: 6814167 (Open Web Text)

Max Activating Token Index: 9

Text #5

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.8234. Min Act: -0.1700

Data Index: 1937623 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.8287. Min Act: -0.1700

Data Index: 3478107 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.8253. Min Act: -0.1700

Data Index: 8650959 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.8111. Min Act: -0.1700

Data Index: 8074978 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.8011. Min Act: -0.1700

Data Index: 7895716 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.7828. Min Act: -0.1700

Data Index: 8091510 (Open Web Text)

Max Activating Token Index: 6

Text #11

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.7784. Min Act: -0.1700

Data Index: 442057 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.7810. Min Act: -0.1700

Data Index: 5182977 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.7894. Min Act: -0.1700

Data Index: 4620279 (Open Web Text)

Max Activating Token Index: 11

Text #14

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.7891. Min Act: -0.1700

Data Index: 2812669 (Open Web Text)

Max Activating Token Index: 7

Text #15

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.7898. Min Act: -0.1700

Data Index: 1506186 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.7712. Min Act: -0.1700

Data Index: 7921219 (Open Web Text)

Max Activating Token Index: 16

Text #17

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.7808. Min Act: -0.1700

Data Index: 3447226 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.7898. Min Act: -0.1700

Data Index: 7512684 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 2.8860. Min Range: -2.8860

Max Act: 2.7688. Min Act: -0.1700

Data Index: 912226 (Open Web Text)

Max Activating Token Index: 6