Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1998 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0624. Min Range: -4.0624

Max Act: 4.0624. Min Act: -0.1700

Data Index: 7239335 (Open Web Text)

Max Activating Token Index: 731

Text #1

Max Range: 4.0624. Min Range: -4.0624

Max Act: 4.0315. Min Act: -0.1700

Data Index: 6574442 (Open Web Text)

Max Activating Token Index: 600

Text #2

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.9444. Min Act: -0.1700

Data Index: 1565837 (Open Web Text)

Max Activating Token Index: 174

Text #3

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.9405. Min Act: -0.1700

Data Index: 5242038 (Open Web Text)

Max Activating Token Index: 676

Text #4

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.9247. Min Act: -0.1700

Data Index: 2663160 (Open Web Text)

Max Activating Token Index: 489

Text #5

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8958. Min Act: -0.1700

Data Index: 7117320 (Open Web Text)

Max Activating Token Index: 901

Text #6

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.9053. Min Act: -0.1700

Data Index: 8200359 (Open Web Text)

Max Activating Token Index: 482

Text #7

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.9013. Min Act: -0.1700

Data Index: 4551418 (Open Web Text)

Max Activating Token Index: 548

Text #8

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8823. Min Act: -0.1700

Data Index: 7387998 (Open Web Text)

Max Activating Token Index: 248

Text #9

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8854. Min Act: -0.1700

Data Index: 7387996 (Open Web Text)

Max Activating Token Index: 827

Text #10

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8820. Min Act: -0.1700

Data Index: 394613 (Open Web Text)

Max Activating Token Index: 925

Text #11

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8485. Min Act: -0.1700

Data Index: 7025980 (Open Web Text)

Max Activating Token Index: 389

Text #12

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8378. Min Act: -0.1700

Data Index: 3056953 (Open Web Text)

Max Activating Token Index: 579

Text #13

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8398. Min Act: -0.1700

Data Index: 153638 (Open Web Text)

Max Activating Token Index: 930

Text #14

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8665. Min Act: -0.1700

Data Index: 1923482 (Open Web Text)

Max Activating Token Index: 804

Text #15

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8548. Min Act: -0.1700

Data Index: 7388000 (Open Web Text)

Max Activating Token Index: 519

Text #16

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8651. Min Act: -0.1700

Data Index: 2449480 (Open Web Text)

Max Activating Token Index: 357

Text #17

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8590. Min Act: -0.1700

Data Index: 2535467 (Open Web Text)

Max Activating Token Index: 307

Text #18

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8631. Min Act: -0.1700

Data Index: 8572045 (Open Web Text)

Max Activating Token Index: 28

Text #19

Max Range: 4.0624. Min Range: -4.0624

Max Act: 3.8372. Min Act: -0.1700

Data Index: 4137604 (Open Web Text)

Max Activating Token Index: 704