Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 633 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.9469. Min Act: -0.1695

Data Index: 4343799 (Open Web Text)

Max Activating Token Index: 66

Text #1

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.8772. Min Act: -0.1696

Data Index: 965495 (Open Web Text)

Max Activating Token Index: 13

Text #2

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.8366. Min Act: -0.1700

Data Index: 446421 (Open Web Text)

Max Activating Token Index: 16

Text #3

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7678. Min Act: -0.1699

Data Index: 2480380 (Open Web Text)

Max Activating Token Index: 7

Text #4

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7840. Min Act: -0.1700

Data Index: 8592005 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7918. Min Act: -0.1700

Data Index: 7511505 (Open Web Text)

Max Activating Token Index: 329

Text #6

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7972. Min Act: -0.1696

Data Index: 998259 (Open Web Text)

Max Activating Token Index: 7

Text #7

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7871. Min Act: -0.1696

Data Index: 8702296 (Open Web Text)

Max Activating Token Index: 465

Text #8

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7448. Min Act: -0.1700

Data Index: 8651914 (Open Web Text)

Max Activating Token Index: 264

Text #9

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7515. Min Act: -0.1700

Data Index: 6979915 (Open Web Text)

Max Activating Token Index: 241

Text #10

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7491. Min Act: -0.1699

Data Index: 2747533 (Open Web Text)

Max Activating Token Index: 10

Text #11

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7388. Min Act: -0.1700

Data Index: 1432511 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7741. Min Act: -0.1700

Data Index: 557315 (Open Web Text)

Max Activating Token Index: 32

Text #13

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7522. Min Act: -0.1686

Data Index: 5820041 (Open Web Text)

Max Activating Token Index: 8

Text #14

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7687. Min Act: -0.1700

Data Index: 4216190 (Open Web Text)

Max Activating Token Index: 24

Text #15

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7010. Min Act: -0.1696

Data Index: 7106576 (Open Web Text)

Max Activating Token Index: 296

Text #16

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7386. Min Act: -0.1699

Data Index: 7265118 (Open Web Text)

Max Activating Token Index: 13

Text #17

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.6958. Min Act: -0.1700

Data Index: 7453488 (Open Web Text)

Max Activating Token Index: 18

Text #18

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7235. Min Act: -0.1688

Data Index: 3945526 (Open Web Text)

Max Activating Token Index: 755

Text #19

Max Range: 4.9469. Min Range: -4.9469

Max Act: 4.7342. Min Act: -0.1699

Data Index: 457673 (Open Web Text)

Max Activating Token Index: 29