Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 15 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.5260. Min Act: -0.1700

Data Index: 7800837 (Open Web Text)

Max Activating Token Index: 109

Text #1

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.4576. Min Act: -0.1700

Data Index: 3159443 (Open Web Text)

Max Activating Token Index: 615

Text #2

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.4461. Min Act: -0.1700

Data Index: 356442 (Open Web Text)

Max Activating Token Index: 762

Text #3

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.4424. Min Act: -0.1700

Data Index: 3536454 (Open Web Text)

Max Activating Token Index: 908

Text #4

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.4214. Min Act: -0.1700

Data Index: 988571 (Open Web Text)

Max Activating Token Index: 527

Text #5

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.3801. Min Act: -0.1700

Data Index: 1900092 (Open Web Text)

Max Activating Token Index: 851

Text #6

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.3482. Min Act: -0.1700

Data Index: 1762793 (Open Web Text)

Max Activating Token Index: 178

Text #7

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.3020. Min Act: -0.1700

Data Index: 7436937 (Open Web Text)

Max Activating Token Index: 451

Text #8

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.3113. Min Act: -0.1700

Data Index: 859026 (Open Web Text)

Max Activating Token Index: 668

Text #9

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.3112. Min Act: -0.1700

Data Index: 4439410 (Open Web Text)

Max Activating Token Index: 747

Text #10

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.3214. Min Act: -0.1700

Data Index: 2890702 (Open Web Text)

Max Activating Token Index: 322

Text #11

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.2911. Min Act: -0.1700

Data Index: 3412153 (Open Web Text)

Max Activating Token Index: 167

Text #12

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.2945. Min Act: -0.1700

Data Index: 1181881 (Open Web Text)

Max Activating Token Index: 859

Text #13

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.2846. Min Act: -0.1700

Data Index: 2160403 (Open Web Text)

Max Activating Token Index: 689

Text #14

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.2683. Min Act: -0.1700

Data Index: 4004704 (Open Web Text)

Max Activating Token Index: 1021

Text #15

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.2762. Min Act: -0.1700

Data Index: 3582569 (Open Web Text)

Max Activating Token Index: 372

Text #16

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.2420. Min Act: -0.1700

Data Index: 4969926 (Open Web Text)

Max Activating Token Index: 104

Text #17

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.2567. Min Act: -0.1700

Data Index: 7353206 (Open Web Text)

Max Activating Token Index: 103

Text #18

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.2686. Min Act: -0.1700

Data Index: 4771074 (Open Web Text)

Max Activating Token Index: 750

Text #19

Max Range: 3.5260. Min Range: -3.5260

Max Act: 3.2435. Min Act: -0.1700

Data Index: 4508286 (Open Web Text)

Max Activating Token Index: 453