Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1164 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0223. Min Range: -3.0223

Max Act: 3.0223. Min Act: -0.1700

Data Index: 5451863 (Open Web Text)

Max Activating Token Index: 177

Text #1

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.9223. Min Act: -0.1700

Data Index: 2140543 (Open Web Text)

Max Activating Token Index: 566

Text #2

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.9647. Min Act: -0.1700

Data Index: 5261049 (Open Web Text)

Max Activating Token Index: 456

Text #3

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.8227. Min Act: -0.1700

Data Index: 6348865 (Open Web Text)

Max Activating Token Index: 376

Text #4

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7829. Min Act: -0.1700

Data Index: 4268806 (Open Web Text)

Max Activating Token Index: 640

Text #5

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7738. Min Act: -0.1700

Data Index: 3695712 (Open Web Text)

Max Activating Token Index: 128

Text #6

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7635. Min Act: -0.1700

Data Index: 2351063 (Open Web Text)

Max Activating Token Index: 204

Text #7

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7997. Min Act: -0.1700

Data Index: 4255275 (Open Web Text)

Max Activating Token Index: 602

Text #8

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7624. Min Act: -0.1700

Data Index: 8410135 (Open Web Text)

Max Activating Token Index: 431

Text #9

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7372. Min Act: -0.1700

Data Index: 6850993 (Open Web Text)

Max Activating Token Index: 440

Text #10

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7374. Min Act: -0.1700

Data Index: 1559107 (Open Web Text)

Max Activating Token Index: 500

Text #11

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7326. Min Act: -0.1700

Data Index: 3859481 (Open Web Text)

Max Activating Token Index: 174

Text #12

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7236. Min Act: -0.1700

Data Index: 411746 (Open Web Text)

Max Activating Token Index: 44

Text #13

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7099. Min Act: -0.1700

Data Index: 4113960 (Open Web Text)

Max Activating Token Index: 851

Text #14

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7221. Min Act: -0.1700

Data Index: 3258138 (Open Web Text)

Max Activating Token Index: 422

Text #15

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7058. Min Act: -0.1700

Data Index: 1781025 (Open Web Text)

Max Activating Token Index: 324

Text #16

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7181. Min Act: -0.1700

Data Index: 5874658 (Open Web Text)

Max Activating Token Index: 446

Text #17

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7204. Min Act: -0.1700

Data Index: 2094967 (Open Web Text)

Max Activating Token Index: 83

Text #18

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.6948. Min Act: -0.1700

Data Index: 106074 (Open Web Text)

Max Activating Token Index: 174

Text #19

Max Range: 3.0223. Min Range: -3.0223

Max Act: 2.7071. Min Act: -0.1700

Data Index: 4378404 (Open Web Text)

Max Activating Token Index: 794