Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1831 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.7889. Min Act: -0.1700

Data Index: 7227749 (Open Web Text)

Max Activating Token Index: 637

Text #1

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.8040. Min Act: -0.1700

Data Index: 731973 (Open Web Text)

Max Activating Token Index: 128

Text #2

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.7507. Min Act: -0.1700

Data Index: 655505 (Open Web Text)

Max Activating Token Index: 540

Text #3

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.7100. Min Act: -0.1700

Data Index: 3857424 (Open Web Text)

Max Activating Token Index: 747

Text #4

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.7056. Min Act: -0.1700

Data Index: 4937217 (Open Web Text)

Max Activating Token Index: 445

Text #5

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.7049. Min Act: -0.1700

Data Index: 3020920 (Open Web Text)

Max Activating Token Index: 923

Text #6

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.6909. Min Act: -0.1700

Data Index: 1389872 (Open Web Text)

Max Activating Token Index: 518

Text #7

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.7139. Min Act: -0.1700

Data Index: 1038452 (Open Web Text)

Max Activating Token Index: 700

Text #8

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.7079. Min Act: -0.1700

Data Index: 1486215 (Open Web Text)

Max Activating Token Index: 554

Text #9

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.7012. Min Act: -0.1700

Data Index: 3252413 (Open Web Text)

Max Activating Token Index: 1005

Text #10

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.7208. Min Act: -0.1700

Data Index: 1871748 (Open Web Text)

Max Activating Token Index: 635

Text #11

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.6698. Min Act: -0.1700

Data Index: 7347678 (Open Web Text)

Max Activating Token Index: 853

Text #12

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.6714. Min Act: -0.1700

Data Index: 1167612 (Open Web Text)

Max Activating Token Index: 132

Text #13

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.6684. Min Act: -0.1700

Data Index: 7334471 (Open Web Text)

Max Activating Token Index: 771

Text #14

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.6671. Min Act: -0.1700

Data Index: 7092356 (Open Web Text)

Max Activating Token Index: 174

Text #15

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.6644. Min Act: -0.1700

Data Index: 4652121 (Open Web Text)

Max Activating Token Index: 102

Text #16

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.6786. Min Act: -0.1700

Data Index: 836285 (Open Web Text)

Max Activating Token Index: 564

Text #17

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.6381. Min Act: -0.1700

Data Index: 3595398 (Open Web Text)

Max Activating Token Index: 537

Text #18

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.6492. Min Act: -0.1700

Data Index: 6953456 (Open Web Text)

Max Activating Token Index: 908

Text #19

Max Range: 4.8040. Min Range: -4.8040

Max Act: 4.6619. Min Act: -0.1700

Data Index: 10621 (Open Web Text)

Max Activating Token Index: 530