Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 528 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7914. Min Act: -0.1700

Data Index: 2770445 (Open Web Text)

Max Activating Token Index: 878

Text #1

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7648. Min Act: -0.1700

Data Index: 4416512 (Open Web Text)

Max Activating Token Index: 887

Text #2

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7682. Min Act: -0.1700

Data Index: 6626270 (Open Web Text)

Max Activating Token Index: 864

Text #3

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7615. Min Act: -0.1700

Data Index: 2698869 (Open Web Text)

Max Activating Token Index: 868

Text #4

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7579. Min Act: -0.1700

Data Index: 936347 (Open Web Text)

Max Activating Token Index: 873

Text #5

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7537. Min Act: -0.1700

Data Index: 2251676 (Open Web Text)

Max Activating Token Index: 889

Text #6

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7447. Min Act: -0.1700

Data Index: 3901472 (Open Web Text)

Max Activating Token Index: 878

Text #7

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7445. Min Act: -0.1700

Data Index: 3643099 (Open Web Text)

Max Activating Token Index: 879

Text #8

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7571. Min Act: -0.1700

Data Index: 4638530 (Open Web Text)

Max Activating Token Index: 890

Text #9

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7579. Min Act: -0.1700

Data Index: 7538759 (Open Web Text)

Max Activating Token Index: 880

Text #10

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7510. Min Act: -0.1700

Data Index: 6388012 (Open Web Text)

Max Activating Token Index: 885

Text #11

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7491. Min Act: -0.1700

Data Index: 1982990 (Open Web Text)

Max Activating Token Index: 856

Text #12

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7454. Min Act: -0.1700

Data Index: 6893312 (Open Web Text)

Max Activating Token Index: 872

Text #13

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7441. Min Act: -0.1700

Data Index: 2485071 (Open Web Text)

Max Activating Token Index: 881

Text #14

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7397. Min Act: -0.1700

Data Index: 212065 (Open Web Text)

Max Activating Token Index: 886

Text #15

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7402. Min Act: -0.1700

Data Index: 2630192 (Open Web Text)

Max Activating Token Index: 879

Text #16

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7454. Min Act: -0.1700

Data Index: 458361 (Open Web Text)

Max Activating Token Index: 879

Text #17

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7446. Min Act: -0.1700

Data Index: 6248139 (Open Web Text)

Max Activating Token Index: 891

Text #18

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7303. Min Act: -0.1700

Data Index: 559486 (Open Web Text)

Max Activating Token Index: 883

Text #19

Max Range: 0.7914. Min Range: -0.7914

Max Act: 0.7366. Min Act: -0.1700

Data Index: 625093 (Open Web Text)

Max Activating Token Index: 877