Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 224 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7914. Min Act: -0.1700

Data Index: 5394857 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.8162. Min Act: -0.1700

Data Index: 3201632 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7917. Min Act: -0.1700

Data Index: 5672585 (Open Web Text)

Max Activating Token Index: 10

Text #3

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7606. Min Act: -0.1700

Data Index: 4407044 (Open Web Text)

Max Activating Token Index: 6

Text #4

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7711. Min Act: -0.1700

Data Index: 4724977 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7820. Min Act: -0.1700

Data Index: 3130842 (Open Web Text)

Max Activating Token Index: 7

Text #6

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7376. Min Act: -0.1700

Data Index: 4404244 (Open Web Text)

Max Activating Token Index: 7

Text #7

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7376. Min Act: -0.1700

Data Index: 3281286 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7325. Min Act: -0.1700

Data Index: 8614601 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7432. Min Act: -0.1700

Data Index: 8500326 (Open Web Text)

Max Activating Token Index: 7

Text #10

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7475. Min Act: -0.1700

Data Index: 8226448 (Open Web Text)

Max Activating Token Index: 6

Text #11

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7542. Min Act: -0.1700

Data Index: 7467830 (Open Web Text)

Max Activating Token Index: 12

Text #12

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7370. Min Act: -0.1700

Data Index: 6727856 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7290. Min Act: -0.1700

Data Index: 5459259 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7346. Min Act: -0.1700

Data Index: 863940 (Open Web Text)

Max Activating Token Index: 3

Text #15

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.6997. Min Act: -0.1700

Data Index: 3210418 (Open Web Text)

Max Activating Token Index: 6

Text #16

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7351. Min Act: -0.1700

Data Index: 3970995 (Open Web Text)

Max Activating Token Index: 858

Text #17

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.7203. Min Act: -0.1700

Data Index: 4123484 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.6999. Min Act: -0.1700

Data Index: 4204190 (Open Web Text)

Max Activating Token Index: 3

Text #19

Max Range: 3.8162. Min Range: -3.8162

Max Act: 3.6871. Min Act: -0.1700

Data Index: 4234203 (Open Web Text)

Max Activating Token Index: 1010