Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 507 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.4676. Min Act: -0.1700

Data Index: 527492 (Open Web Text)

Max Activating Token Index: 87

Text #1

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.4130. Min Act: -0.1700

Data Index: 5135255 (Open Web Text)

Max Activating Token Index: 143

Text #2

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.4097. Min Act: -0.1700

Data Index: 8462377 (Open Web Text)

Max Activating Token Index: 535

Text #3

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3917. Min Act: -0.1700

Data Index: 2243209 (Open Web Text)

Max Activating Token Index: 163

Text #4

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3808. Min Act: -0.1700

Data Index: 6277180 (Open Web Text)

Max Activating Token Index: 955

Text #5

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3647. Min Act: -0.1700

Data Index: 7550901 (Open Web Text)

Max Activating Token Index: 225

Text #6

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3679. Min Act: -0.1700

Data Index: 1616610 (Open Web Text)

Max Activating Token Index: 248

Text #7

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3616. Min Act: -0.1700

Data Index: 6056120 (Open Web Text)

Max Activating Token Index: 298

Text #8

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3717. Min Act: -0.1700

Data Index: 1493620 (Open Web Text)

Max Activating Token Index: 64

Text #9

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3588. Min Act: -0.1700

Data Index: 8285288 (Open Web Text)

Max Activating Token Index: 746

Text #10

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3534. Min Act: -0.1700

Data Index: 811255 (Open Web Text)

Max Activating Token Index: 353

Text #11

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3557. Min Act: -0.1700

Data Index: 5451043 (Open Web Text)

Max Activating Token Index: 473

Text #12

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3518. Min Act: -0.1700

Data Index: 243587 (Open Web Text)

Max Activating Token Index: 560

Text #13

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3366. Min Act: -0.1700

Data Index: 4574896 (Open Web Text)

Max Activating Token Index: 43

Text #14

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3257. Min Act: -0.1700

Data Index: 7859345 (Open Web Text)

Max Activating Token Index: 333

Text #15

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3285. Min Act: -0.1700

Data Index: 2700901 (Open Web Text)

Max Activating Token Index: 324

Text #16

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3451. Min Act: -0.1700

Data Index: 5508367 (Open Web Text)

Max Activating Token Index: 32

Text #17

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3313. Min Act: -0.1700

Data Index: 7864394 (Open Web Text)

Max Activating Token Index: 508

Text #18

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3098. Min Act: -0.1700

Data Index: 5375574 (Open Web Text)

Max Activating Token Index: 26

Text #19

Max Range: 3.4676. Min Range: -3.4676

Max Act: 3.3312. Min Act: -0.1700

Data Index: 4081257 (Open Web Text)

Max Activating Token Index: 106