Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2625 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.8875. Min Act: -0.1700

Data Index: 7646725 (Open Web Text)

Max Activating Token Index: 927

Text #1

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7566. Min Act: -0.1700

Data Index: 4946162 (Open Web Text)

Max Activating Token Index: 464

Text #2

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.8528. Min Act: -0.1700

Data Index: 3580543 (Open Web Text)

Max Activating Token Index: 744

Text #3

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7314. Min Act: -0.1700

Data Index: 6909203 (Open Web Text)

Max Activating Token Index: 543

Text #4

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7819. Min Act: -0.1700

Data Index: 5880190 (Open Web Text)

Max Activating Token Index: 60

Text #5

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7878. Min Act: -0.1700

Data Index: 872392 (Open Web Text)

Max Activating Token Index: 109

Text #6

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7481. Min Act: -0.1700

Data Index: 4523025 (Open Web Text)

Max Activating Token Index: 179

Text #7

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7860. Min Act: -0.1700

Data Index: 290886 (Open Web Text)

Max Activating Token Index: 414

Text #8

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7329. Min Act: -0.1700

Data Index: 4551904 (Open Web Text)

Max Activating Token Index: 375

Text #9

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7696. Min Act: -0.1700

Data Index: 3189443 (Open Web Text)

Max Activating Token Index: 487

Text #10

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7208. Min Act: -0.1700

Data Index: 8239534 (Open Web Text)

Max Activating Token Index: 88

Text #11

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.6864. Min Act: -0.1700

Data Index: 8230832 (Open Web Text)

Max Activating Token Index: 524

Text #12

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7217. Min Act: -0.1700

Data Index: 5795335 (Open Web Text)

Max Activating Token Index: 383

Text #13

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7357. Min Act: -0.1700

Data Index: 3355754 (Open Web Text)

Max Activating Token Index: 434

Text #14

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7157. Min Act: -0.1700

Data Index: 30523 (Open Web Text)

Max Activating Token Index: 781

Text #15

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7034. Min Act: -0.1700

Data Index: 3288058 (Open Web Text)

Max Activating Token Index: 92

Text #16

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.6585. Min Act: -0.1700

Data Index: 4784995 (Open Web Text)

Max Activating Token Index: 225

Text #17

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.6899. Min Act: -0.1700

Data Index: 7542037 (Open Web Text)

Max Activating Token Index: 238

Text #18

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.6520. Min Act: -0.1700

Data Index: 2977549 (Open Web Text)

Max Activating Token Index: 114

Text #19

Max Range: 3.8875. Min Range: -3.8875

Max Act: 3.7351. Min Act: -0.1700

Data Index: 7664949 (Open Web Text)

Max Activating Token Index: 333