Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 941 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.9083. Min Act: -0.1700

Data Index: 910671 (Open Web Text)

Max Activating Token Index: 473

Text #1

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.6694. Min Act: -0.1700

Data Index: 1249510 (Open Web Text)

Max Activating Token Index: 438

Text #2

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.5943. Min Act: -0.1700

Data Index: 3743039 (Open Web Text)

Max Activating Token Index: 965

Text #3

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.5383. Min Act: -0.1700

Data Index: 992980 (Open Web Text)

Max Activating Token Index: 546

Text #4

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.5423. Min Act: -0.1700

Data Index: 7995844 (Open Web Text)

Max Activating Token Index: 612

Text #5

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.5380. Min Act: -0.1700

Data Index: 2725486 (Open Web Text)

Max Activating Token Index: 262

Text #6

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.5429. Min Act: -0.1700

Data Index: 699446 (Open Web Text)

Max Activating Token Index: 486

Text #7

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.5022. Min Act: -0.1700

Data Index: 1147993 (Open Web Text)

Max Activating Token Index: 140

Text #8

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.4964. Min Act: -0.1700

Data Index: 5582516 (Open Web Text)

Max Activating Token Index: 55

Text #9

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.4959. Min Act: -0.1700

Data Index: 8229007 (Open Web Text)

Max Activating Token Index: 443

Text #10

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.4574. Min Act: -0.1700

Data Index: 3808208 (Open Web Text)

Max Activating Token Index: 470

Text #11

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.4861. Min Act: -0.1700

Data Index: 434449 (Open Web Text)

Max Activating Token Index: 941

Text #12

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.4768. Min Act: -0.1700

Data Index: 2387889 (Open Web Text)

Max Activating Token Index: 531

Text #13

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.4570. Min Act: -0.1700

Data Index: 5333742 (Open Web Text)

Max Activating Token Index: 488

Text #14

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.4517. Min Act: -0.1700

Data Index: 5363878 (Open Web Text)

Max Activating Token Index: 965

Text #15

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.4774. Min Act: -0.1700

Data Index: 3491465 (Open Web Text)

Max Activating Token Index: 557

Text #16

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.4433. Min Act: -0.1700

Data Index: 1919579 (Open Web Text)

Max Activating Token Index: 919

Text #17

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.4395. Min Act: -0.1700

Data Index: 1441588 (Open Web Text)

Max Activating Token Index: 454

Text #18

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.4308. Min Act: -0.1700

Data Index: 399789 (Open Web Text)

Max Activating Token Index: 731

Text #19

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.4351. Min Act: -0.1700

Data Index: 1025344 (Open Web Text)

Max Activating Token Index: 428