Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2131 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.8451. Min Act: -0.1700

Data Index: 7158986 (Open Web Text)

Max Activating Token Index: 542

Text #1

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.8480. Min Act: -0.1700

Data Index: 444642 (Open Web Text)

Max Activating Token Index: 519

Text #2

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7914. Min Act: -0.1700

Data Index: 1587240 (Open Web Text)

Max Activating Token Index: 565

Text #3

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.8004. Min Act: -0.1700

Data Index: 4819718 (Open Web Text)

Max Activating Token Index: 520

Text #4

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7971. Min Act: -0.1700

Data Index: 7883345 (Open Web Text)

Max Activating Token Index: 525

Text #5

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7967. Min Act: -0.1700

Data Index: 1046156 (Open Web Text)

Max Activating Token Index: 531

Text #6

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7982. Min Act: -0.1700

Data Index: 7820241 (Open Web Text)

Max Activating Token Index: 545

Text #7

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.8026. Min Act: -0.1700

Data Index: 5579038 (Open Web Text)

Max Activating Token Index: 551

Text #8

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7919. Min Act: -0.1700

Data Index: 3914419 (Open Web Text)

Max Activating Token Index: 529

Text #9

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7915. Min Act: -0.1700

Data Index: 5154909 (Open Web Text)

Max Activating Token Index: 557

Text #10

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7896. Min Act: -0.1700

Data Index: 5309479 (Open Web Text)

Max Activating Token Index: 553

Text #11

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7919. Min Act: -0.1700

Data Index: 336142 (Open Web Text)

Max Activating Token Index: 548

Text #12

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7852. Min Act: -0.1700

Data Index: 6668331 (Open Web Text)

Max Activating Token Index: 503

Text #13

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7887. Min Act: -0.1700

Data Index: 932461 (Open Web Text)

Max Activating Token Index: 527

Text #14

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7925. Min Act: -0.1700

Data Index: 1140701 (Open Web Text)

Max Activating Token Index: 531

Text #15

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7814. Min Act: -0.1700

Data Index: 2127736 (Open Web Text)

Max Activating Token Index: 531

Text #16

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7926. Min Act: -0.1700

Data Index: 6407298 (Open Web Text)

Max Activating Token Index: 536

Text #17

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7939. Min Act: -0.1700

Data Index: 7883349 (Open Web Text)

Max Activating Token Index: 528

Text #18

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7800. Min Act: -0.1700

Data Index: 3426667 (Open Web Text)

Max Activating Token Index: 536

Text #19

Max Range: 0.8480. Min Range: -0.8480

Max Act: 0.7896. Min Act: -0.1700

Data Index: 352444 (Open Web Text)

Max Activating Token Index: 488