Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2552 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.7535. Min Act: -0.1700

Data Index: 4063191 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.7280. Min Act: -0.1700

Data Index: 7460682 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.7237. Min Act: -0.1700

Data Index: 2031963 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.7150. Min Act: -0.1700

Data Index: 7414056 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.6402. Min Act: -0.1700

Data Index: 4252782 (Open Web Text)

Max Activating Token Index: 13

Text #5

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.6508. Min Act: -0.1700

Data Index: 7352735 (Open Web Text)

Max Activating Token Index: 17

Text #6

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.6262. Min Act: -0.1700

Data Index: 6723440 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.6222. Min Act: -0.1700

Data Index: 4763602 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.6302. Min Act: -0.1700

Data Index: 1290340 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.6222. Min Act: -0.1700

Data Index: 2377918 (Open Web Text)

Max Activating Token Index: 2

Text #10

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.6189. Min Act: -0.1700

Data Index: 2907344 (Open Web Text)

Max Activating Token Index: 641

Text #11

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.6048. Min Act: -0.1700

Data Index: 3059015 (Open Web Text)

Max Activating Token Index: 10

Text #12

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.5947. Min Act: -0.1700

Data Index: 2492378 (Open Web Text)

Max Activating Token Index: 764

Text #13

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.6043. Min Act: -0.1700

Data Index: 8686973 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.5949. Min Act: -0.1700

Data Index: 6666421 (Open Web Text)

Max Activating Token Index: 3

Text #15

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.5917. Min Act: -0.1700

Data Index: 2792972 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.5943. Min Act: -0.1700

Data Index: 5643580 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.5811. Min Act: -0.1700

Data Index: 7224517 (Open Web Text)

Max Activating Token Index: 794

Text #18

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.5781. Min Act: -0.1700

Data Index: 6086970 (Open Web Text)

Max Activating Token Index: 7

Text #19

Max Range: 2.7535. Min Range: -2.7535

Max Act: 2.5943. Min Act: -0.1700

Data Index: 6102522 (Open Web Text)

Max Activating Token Index: 5