Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 609 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.8912. Min Act: -0.1700

Data Index: 1170765 (Open Web Text)

Max Activating Token Index: 19

Text #1

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.8969. Min Act: -0.1700

Data Index: 3340015 (Open Web Text)

Max Activating Token Index: 668

Text #2

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.8364. Min Act: -0.1700

Data Index: 32803 (Open Web Text)

Max Activating Token Index: 9

Text #3

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.8336. Min Act: -0.1700

Data Index: 4104689 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.8130. Min Act: -0.1700

Data Index: 8385076 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.8321. Min Act: -0.1700

Data Index: 3972976 (Open Web Text)

Max Activating Token Index: 250

Text #6

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.7788. Min Act: -0.1700

Data Index: 7234865 (Open Web Text)

Max Activating Token Index: 18

Text #7

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.7532. Min Act: -0.1700

Data Index: 6159034 (Open Web Text)

Max Activating Token Index: 152

Text #8

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.7340. Min Act: -0.1700

Data Index: 575402 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.7143. Min Act: -0.1700

Data Index: 839978 (Open Web Text)

Max Activating Token Index: 54

Text #10

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.7323. Min Act: -0.1700

Data Index: 2070006 (Open Web Text)

Max Activating Token Index: 6

Text #11

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.7087. Min Act: -0.1700

Data Index: 6952061 (Open Web Text)

Max Activating Token Index: 610

Text #12

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.7045. Min Act: -0.1700

Data Index: 3624313 (Open Web Text)

Max Activating Token Index: 188

Text #13

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.6814. Min Act: -0.1700

Data Index: 281687 (Open Web Text)

Max Activating Token Index: 8

Text #14

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.7056. Min Act: -0.1700

Data Index: 2426313 (Open Web Text)

Max Activating Token Index: 483

Text #15

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.6736. Min Act: -0.1700

Data Index: 6174935 (Open Web Text)

Max Activating Token Index: 712

Text #16

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.7015. Min Act: -0.1700

Data Index: 2040533 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.6952. Min Act: -0.1700

Data Index: 2677291 (Open Web Text)

Max Activating Token Index: 32

Text #18

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.6835. Min Act: -0.1700

Data Index: 5935336 (Open Web Text)

Max Activating Token Index: 662

Text #19

Max Range: 3.8969. Min Range: -3.8969

Max Act: 3.6470. Min Act: -0.1700

Data Index: 1278975 (Open Web Text)

Max Activating Token Index: 401