Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1208 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0928. Min Range: -3.0928

Max Act: 3.0876. Min Act: -0.1700

Data Index: 5473709 (Open Web Text)

Max Activating Token Index: 15

Text #1

Max Range: 3.0928. Min Range: -3.0928

Max Act: 3.0928. Min Act: -0.1700

Data Index: 2498121 (Open Web Text)

Max Activating Token Index: 321

Text #2

Max Range: 3.0928. Min Range: -3.0928

Max Act: 3.0596. Min Act: -0.1700

Data Index: 5589760 (Open Web Text)

Max Activating Token Index: 12

Text #3

Max Range: 3.0928. Min Range: -3.0928

Max Act: 3.0398. Min Act: -0.1700

Data Index: 830866 (Open Web Text)

Max Activating Token Index: 16

Text #4

Max Range: 3.0928. Min Range: -3.0928

Max Act: 3.0382. Min Act: -0.1700

Data Index: 5354427 (Open Web Text)

Max Activating Token Index: 9

Text #5

Max Range: 3.0928. Min Range: -3.0928

Max Act: 3.0474. Min Act: -0.1700

Data Index: 5172421 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 3.0928. Min Range: -3.0928

Max Act: 3.0205. Min Act: -0.1700

Data Index: 6852881 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 3.0928. Min Range: -3.0928

Max Act: 3.0289. Min Act: -0.1700

Data Index: 7659544 (Open Web Text)

Max Activating Token Index: 12

Text #8

Max Range: 3.0928. Min Range: -3.0928

Max Act: 3.0218. Min Act: -0.1700

Data Index: 1259578 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 3.0928. Min Range: -3.0928

Max Act: 3.0137. Min Act: -0.1700

Data Index: 3812456 (Open Web Text)

Max Activating Token Index: 12

Text #10

Max Range: 3.0928. Min Range: -3.0928

Max Act: 2.9861. Min Act: -0.1700

Data Index: 5758997 (Open Web Text)

Max Activating Token Index: 12

Text #11

Max Range: 3.0928. Min Range: -3.0928

Max Act: 2.9795. Min Act: -0.1700

Data Index: 2953100 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 3.0928. Min Range: -3.0928

Max Act: 2.9788. Min Act: -0.1700

Data Index: 1645214 (Open Web Text)

Max Activating Token Index: 15

Text #13

Max Range: 3.0928. Min Range: -3.0928

Max Act: 2.9593. Min Act: -0.1700

Data Index: 681135 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 3.0928. Min Range: -3.0928

Max Act: 2.9623. Min Act: -0.1700

Data Index: 83647 (Open Web Text)

Max Activating Token Index: 16

Text #15

Max Range: 3.0928. Min Range: -3.0928

Max Act: 2.9670. Min Act: -0.1700

Data Index: 6800027 (Open Web Text)

Max Activating Token Index: 14

Text #16

Max Range: 3.0928. Min Range: -3.0928

Max Act: 2.9606. Min Act: -0.1700

Data Index: 1469038 (Open Web Text)

Max Activating Token Index: 16

Text #17

Max Range: 3.0928. Min Range: -3.0928

Max Act: 2.9421. Min Act: -0.1700

Data Index: 3643162 (Open Web Text)

Max Activating Token Index: 14

Text #18

Max Range: 3.0928. Min Range: -3.0928

Max Act: 2.9553. Min Act: -0.1700

Data Index: 2692100 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 3.0928. Min Range: -3.0928

Max Act: 2.9374. Min Act: -0.1700

Data Index: 1404768 (Open Web Text)

Max Activating Token Index: 4