Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1811 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.9686. Min Act: -0.1700

Data Index: 3656477 (Open Web Text)

Max Activating Token Index: 871

Text #1

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.9026. Min Act: -0.1700

Data Index: 1288300 (Open Web Text)

Max Activating Token Index: 80

Text #2

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.7891. Min Act: -0.1700

Data Index: 7919996 (Open Web Text)

Max Activating Token Index: 53

Text #3

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.7736. Min Act: -0.1700

Data Index: 7100364 (Open Web Text)

Max Activating Token Index: 398

Text #4

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.7418. Min Act: -0.1700

Data Index: 2765142 (Open Web Text)

Max Activating Token Index: 428

Text #5

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.7121. Min Act: -0.1700

Data Index: 2830262 (Open Web Text)

Max Activating Token Index: 184

Text #6

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.7213. Min Act: -0.1700

Data Index: 8395814 (Open Web Text)

Max Activating Token Index: 548

Text #7

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.7282. Min Act: -0.1700

Data Index: 5769677 (Open Web Text)

Max Activating Token Index: 946

Text #8

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.7485. Min Act: -0.1700

Data Index: 6705175 (Open Web Text)

Max Activating Token Index: 459

Text #9

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.6843. Min Act: -0.1700

Data Index: 1090052 (Open Web Text)

Max Activating Token Index: 304

Text #10

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.6868. Min Act: -0.1700

Data Index: 7179700 (Open Web Text)

Max Activating Token Index: 795

Text #11

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.6281. Min Act: -0.1700

Data Index: 1220253 (Open Web Text)

Max Activating Token Index: 808

Text #12

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.6218. Min Act: -0.1700

Data Index: 1298150 (Open Web Text)

Max Activating Token Index: 795

Text #13

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.6511. Min Act: -0.1700

Data Index: 6872481 (Open Web Text)

Max Activating Token Index: 937

Text #14

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.6463. Min Act: -0.1700

Data Index: 3538117 (Open Web Text)

Max Activating Token Index: 830

Text #15

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.6610. Min Act: -0.1700

Data Index: 5421092 (Open Web Text)

Max Activating Token Index: 728

Text #16

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.6222. Min Act: -0.1700

Data Index: 4304483 (Open Web Text)

Max Activating Token Index: 638

Text #17

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.5847. Min Act: -0.1700

Data Index: 7786087 (Open Web Text)

Max Activating Token Index: 667

Text #18

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.6004. Min Act: -0.1700

Data Index: 3198346 (Open Web Text)

Max Activating Token Index: 365

Text #19

Max Range: 4.9686. Min Range: -4.9686

Max Act: 4.6011. Min Act: -0.1700

Data Index: 2125173 (Open Web Text)

Max Activating Token Index: 298