Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2594 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.8101. Min Act: -0.1700

Data Index: 1176694 (Open Web Text)

Max Activating Token Index: 2

Text #1

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.8161. Min Act: -0.1700

Data Index: 2147992 (Open Web Text)

Max Activating Token Index: 2

Text #2

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.8118. Min Act: -0.1700

Data Index: 7529387 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.8033. Min Act: -0.1700

Data Index: 7811207 (Open Web Text)

Max Activating Token Index: 7

Text #4

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7940. Min Act: -0.1700

Data Index: 1875995 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.8026. Min Act: -0.1700

Data Index: 8175695 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.8014. Min Act: -0.1700

Data Index: 2409151 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.8046. Min Act: -0.1700

Data Index: 5781696 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7804. Min Act: -0.1700

Data Index: 3719231 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7904. Min Act: -0.1700

Data Index: 5914094 (Open Web Text)

Max Activating Token Index: 8

Text #10

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7801. Min Act: -0.1700

Data Index: 5221574 (Open Web Text)

Max Activating Token Index: 3

Text #11

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7898. Min Act: -0.1700

Data Index: 2600653 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7903. Min Act: -0.1700

Data Index: 7940237 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7674. Min Act: -0.1700

Data Index: 3197677 (Open Web Text)

Max Activating Token Index: 2

Text #14

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7636. Min Act: -0.1700

Data Index: 8773013 (Open Web Text)

Max Activating Token Index: 3

Text #15

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7761. Min Act: -0.1700

Data Index: 8333943 (Open Web Text)

Max Activating Token Index: 2

Text #16

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7732. Min Act: -0.1700

Data Index: 8461824 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7661. Min Act: -0.1700

Data Index: 2478531 (Open Web Text)

Max Activating Token Index: 2

Text #18

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7659. Min Act: -0.1700

Data Index: 4313857 (Open Web Text)

Max Activating Token Index: 3

Text #19

Max Range: 2.8161. Min Range: -2.8161

Max Act: 2.7545. Min Act: -0.1700

Data Index: 501969 (Open Web Text)

Max Activating Token Index: 3