Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 983 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.8675. Min Act: -0.1700

Data Index: 5792448 (Open Web Text)

Max Activating Token Index: 478

Text #1

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.8676. Min Act: -0.1700

Data Index: 6630541 (Open Web Text)

Max Activating Token Index: 640

Text #2

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.8240. Min Act: -0.1700

Data Index: 6358087 (Open Web Text)

Max Activating Token Index: 515

Text #3

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.7671. Min Act: -0.1700

Data Index: 1982351 (Open Web Text)

Max Activating Token Index: 753

Text #4

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.7259. Min Act: -0.1700

Data Index: 4607705 (Open Web Text)

Max Activating Token Index: 269

Text #5

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.7171. Min Act: -0.1700

Data Index: 4267273 (Open Web Text)

Max Activating Token Index: 924

Text #6

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.6903. Min Act: -0.1700

Data Index: 7140671 (Open Web Text)

Max Activating Token Index: 335

Text #7

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.6836. Min Act: -0.1700

Data Index: 7504676 (Open Web Text)

Max Activating Token Index: 124

Text #8

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.7033. Min Act: -0.1700

Data Index: 798858 (Open Web Text)

Max Activating Token Index: 997

Text #9

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.6726. Min Act: -0.1700

Data Index: 1253829 (Open Web Text)

Max Activating Token Index: 896

Text #10

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.6245. Min Act: -0.1700

Data Index: 4858407 (Open Web Text)

Max Activating Token Index: 548

Text #11

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.5484. Min Act: -0.1700

Data Index: 3854567 (Open Web Text)

Max Activating Token Index: 524

Text #12

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.5579. Min Act: -0.1700

Data Index: 2046381 (Open Web Text)

Max Activating Token Index: 943

Text #13

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.5538. Min Act: -0.1700

Data Index: 5870231 (Open Web Text)

Max Activating Token Index: 270

Text #14

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.5317. Min Act: -0.1700

Data Index: 5767429 (Open Web Text)

Max Activating Token Index: 888

Text #15

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.5339. Min Act: -0.1700

Data Index: 950849 (Open Web Text)

Max Activating Token Index: 205

Text #16

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.5434. Min Act: -0.1700

Data Index: 8019656 (Open Web Text)

Max Activating Token Index: 795

Text #17

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.5244. Min Act: -0.1700

Data Index: 1393667 (Open Web Text)

Max Activating Token Index: 127

Text #18

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.5108. Min Act: -0.1700

Data Index: 3788167 (Open Web Text)

Max Activating Token Index: 858

Text #19

Max Range: 4.8676. Min Range: -4.8676

Max Act: 4.5252. Min Act: -0.1700

Data Index: 4304316 (Open Web Text)

Max Activating Token Index: 88