Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2606 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7924. Min Act: -0.1700

Data Index: 8473884 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7384. Min Act: -0.1700

Data Index: 2664395 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7377. Min Act: -0.1700

Data Index: 8669098 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7177. Min Act: -0.1700

Data Index: 6605391 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7237. Min Act: -0.1700

Data Index: 3199537 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7015. Min Act: -0.1700

Data Index: 8693866 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7115. Min Act: -0.1700

Data Index: 1837206 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7073. Min Act: -0.1700

Data Index: 1177119 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7158. Min Act: -0.1700

Data Index: 1659986 (Open Web Text)

Max Activating Token Index: 10

Text #9

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.6997. Min Act: -0.1700

Data Index: 246517 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7030. Min Act: -0.1700

Data Index: 6493559 (Open Web Text)

Max Activating Token Index: 2

Text #11

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7121. Min Act: -0.1700

Data Index: 8103938 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7161. Min Act: -0.1700

Data Index: 7131210 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.6948. Min Act: -0.1700

Data Index: 5429622 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.6830. Min Act: -0.1700

Data Index: 4257781 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7006. Min Act: -0.1700

Data Index: 4992701 (Open Web Text)

Max Activating Token Index: 8

Text #16

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.6856. Min Act: -0.1700

Data Index: 6937714 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.7027. Min Act: -0.1700

Data Index: 1998921 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.6989. Min Act: -0.1700

Data Index: 6744407 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 2.7924. Min Range: -2.7924

Max Act: 2.6980. Min Act: -0.1700

Data Index: 2630157 (Open Web Text)

Max Activating Token Index: 13