Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2515 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.3518. Min Act: -0.0625

Data Index: 7869880 (Open Web Text)

Max Activating Token Index: 784

Text #1

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.3376. Min Act: -0.0625

Data Index: 6782384 (Open Web Text)

Max Activating Token Index: 557

Text #2

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.3386. Min Act: -0.0875

Data Index: 2768656 (Open Web Text)

Max Activating Token Index: 82

Text #3

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.3323. Min Act: -0.0878

Data Index: 925588 (Open Web Text)

Max Activating Token Index: 8

Text #4

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.3263. Min Act: -0.0625

Data Index: 1522941 (Open Web Text)

Max Activating Token Index: 974

Text #5

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.3258. Min Act: -0.1234

Data Index: 5755960 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2929. Min Act: -0.1403

Data Index: 815541 (Open Web Text)

Max Activating Token Index: 716

Text #7

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2893. Min Act: -0.0838

Data Index: 104867 (Open Web Text)

Max Activating Token Index: 73

Text #8

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2950. Min Act: -0.1018

Data Index: 3048420 (Open Web Text)

Max Activating Token Index: 396

Text #9

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2898. Min Act: -0.0715

Data Index: 4861344 (Open Web Text)

Max Activating Token Index: 632

Text #10

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2913. Min Act: -0.0625

Data Index: 4817266 (Open Web Text)

Max Activating Token Index: 902

Text #11

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.3013. Min Act: -0.0625

Data Index: 6619234 (Open Web Text)

Max Activating Token Index: 1007

Text #12

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2952. Min Act: -0.0997

Data Index: 6800856 (Open Web Text)

Max Activating Token Index: 6

Text #13

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2908. Min Act: -0.0908

Data Index: 1311214 (Open Web Text)

Max Activating Token Index: 137

Text #14

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2961. Min Act: -0.0722

Data Index: 6727485 (Open Web Text)

Max Activating Token Index: 1005

Text #15

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2815. Min Act: -0.0751

Data Index: 1343273 (Open Web Text)

Max Activating Token Index: 927

Text #16

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2901. Min Act: -0.0720

Data Index: 922497 (Open Web Text)

Max Activating Token Index: 535

Text #17

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2861. Min Act: -0.0908

Data Index: 4730834 (Open Web Text)

Max Activating Token Index: 523

Text #18

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2846. Min Act: -0.0625

Data Index: 6193191 (Open Web Text)

Max Activating Token Index: 396

Text #19

Max Range: 1.3518. Min Range: -1.3518

Max Act: 1.2857. Min Act: -0.1017

Data Index: 25634 (Open Web Text)

Max Activating Token Index: 354