Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1900 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5885. Min Act: -0.1700

Data Index: 2357628 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5797. Min Act: -0.1700

Data Index: 7024347 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5595. Min Act: -0.1700

Data Index: 6240615 (Open Web Text)

Max Activating Token Index: 11

Text #3

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5411. Min Act: -0.1700

Data Index: 418256 (Open Web Text)

Max Activating Token Index: 17

Text #4

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5289. Min Act: -0.1700

Data Index: 6509689 (Open Web Text)

Max Activating Token Index: 7

Text #5

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5340. Min Act: -0.1700

Data Index: 1869602 (Open Web Text)

Max Activating Token Index: 7

Text #6

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5337. Min Act: -0.1700

Data Index: 5041431 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5203. Min Act: -0.1700

Data Index: 4440750 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5153. Min Act: -0.1700

Data Index: 1508432 (Open Web Text)

Max Activating Token Index: 10

Text #9

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5115. Min Act: -0.1700

Data Index: 8736195 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5001. Min Act: -0.1700

Data Index: 8103586 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5022. Min Act: -0.1700

Data Index: 2970356 (Open Web Text)

Max Activating Token Index: 9

Text #12

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5036. Min Act: -0.1700

Data Index: 3090333 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.5069. Min Act: -0.1700

Data Index: 896507 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.4799. Min Act: -0.1700

Data Index: 2364406 (Open Web Text)

Max Activating Token Index: 7

Text #15

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.4847. Min Act: -0.1700

Data Index: 5157903 (Open Web Text)

Max Activating Token Index: 10

Text #16

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.4813. Min Act: -0.1700

Data Index: 8405502 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.4769. Min Act: -0.1700

Data Index: 5825056 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.4583. Min Act: -0.1700

Data Index: 5523125 (Open Web Text)

Max Activating Token Index: 13

Text #19

Max Range: 2.5885. Min Range: -2.5885

Max Act: 2.4544. Min Act: -0.1700

Data Index: 5695734 (Open Web Text)

Max Activating Token Index: 25