Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1384 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.1014. Min Act: -0.1700

Data Index: 414181 (Open Web Text)

Max Activating Token Index: 6

Text #1

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0538. Min Act: -0.1700

Data Index: 631715 (Open Web Text)

Max Activating Token Index: 20

Text #2

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0406. Min Act: -0.1700

Data Index: 7669045 (Open Web Text)

Max Activating Token Index: 21

Text #3

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0412. Min Act: -0.1700

Data Index: 3020313 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0405. Min Act: -0.1700

Data Index: 6994695 (Open Web Text)

Max Activating Token Index: 24

Text #5

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0167. Min Act: -0.1700

Data Index: 5605281 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0191. Min Act: -0.1700

Data Index: 7871437 (Open Web Text)

Max Activating Token Index: 28

Text #7

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0292. Min Act: -0.1700

Data Index: 1538493 (Open Web Text)

Max Activating Token Index: 51

Text #8

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0142. Min Act: -0.1700

Data Index: 4730502 (Open Web Text)

Max Activating Token Index: 57

Text #9

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0129. Min Act: -0.1700

Data Index: 2732996 (Open Web Text)

Max Activating Token Index: 183

Text #10

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0128. Min Act: -0.1700

Data Index: 35892 (Open Web Text)

Max Activating Token Index: 44

Text #11

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0042. Min Act: -0.1700

Data Index: 5019552 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0113. Min Act: -0.1700

Data Index: 3464622 (Open Web Text)

Max Activating Token Index: 119

Text #13

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0194. Min Act: -0.1700

Data Index: 7038130 (Open Web Text)

Max Activating Token Index: 13

Text #14

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0117. Min Act: -0.1700

Data Index: 1068880 (Open Web Text)

Max Activating Token Index: 15

Text #15

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0146. Min Act: -0.1700

Data Index: 5125943 (Open Web Text)

Max Activating Token Index: 14

Text #16

Max Range: 3.1014. Min Range: -3.1014

Max Act: 2.9967. Min Act: -0.1700

Data Index: 1041021 (Open Web Text)

Max Activating Token Index: 7

Text #17

Max Range: 3.1014. Min Range: -3.1014

Max Act: 2.9890. Min Act: -0.1700

Data Index: 1995179 (Open Web Text)

Max Activating Token Index: 11

Text #18

Max Range: 3.1014. Min Range: -3.1014

Max Act: 3.0025. Min Act: -0.1700

Data Index: 6735853 (Open Web Text)

Max Activating Token Index: 916

Text #19

Max Range: 3.1014. Min Range: -3.1014

Max Act: 2.9904. Min Act: -0.1700

Data Index: 6905610 (Open Web Text)

Max Activating Token Index: 10