Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1871 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.8263. Min Act: -0.1700

Data Index: 4035802 (Open Web Text)

Max Activating Token Index: 7

Text #1

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.8295. Min Act: -0.1700

Data Index: 214869 (Open Web Text)

Max Activating Token Index: 9

Text #2

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.8341. Min Act: -0.1700

Data Index: 2974820 (Open Web Text)

Max Activating Token Index: 86

Text #3

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.8045. Min Act: -0.1700

Data Index: 1150773 (Open Web Text)

Max Activating Token Index: 20

Text #4

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7958. Min Act: -0.1700

Data Index: 8473813 (Open Web Text)

Max Activating Token Index: 44

Text #5

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7843. Min Act: -0.1700

Data Index: 4816001 (Open Web Text)

Max Activating Token Index: 12

Text #6

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7911. Min Act: -0.1700

Data Index: 3129247 (Open Web Text)

Max Activating Token Index: 62

Text #7

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7909. Min Act: -0.1700

Data Index: 2745023 (Open Web Text)

Max Activating Token Index: 777

Text #8

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7755. Min Act: -0.1700

Data Index: 4703013 (Open Web Text)

Max Activating Token Index: 676

Text #9

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7766. Min Act: -0.1700

Data Index: 5758502 (Open Web Text)

Max Activating Token Index: 35

Text #10

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7891. Min Act: -0.1700

Data Index: 4573825 (Open Web Text)

Max Activating Token Index: 6

Text #11

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7761. Min Act: -0.1700

Data Index: 4000912 (Open Web Text)

Max Activating Token Index: 549

Text #12

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7867. Min Act: -0.1700

Data Index: 1168766 (Open Web Text)

Max Activating Token Index: 381

Text #13

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7737. Min Act: -0.1700

Data Index: 2799172 (Open Web Text)

Max Activating Token Index: 765

Text #14

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7824. Min Act: -0.1700

Data Index: 5093415 (Open Web Text)

Max Activating Token Index: 7

Text #15

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7816. Min Act: -0.1700

Data Index: 1681704 (Open Web Text)

Max Activating Token Index: 11

Text #16

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7728. Min Act: -0.1700

Data Index: 2805164 (Open Web Text)

Max Activating Token Index: 87

Text #17

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7706. Min Act: -0.1700

Data Index: 5101352 (Open Web Text)

Max Activating Token Index: 13

Text #18

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7737. Min Act: -0.1700

Data Index: 3412090 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 1.8341. Min Range: -1.8341

Max Act: 1.7792. Min Act: -0.1700

Data Index: 4924580 (Open Web Text)

Max Activating Token Index: 766