Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1670 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.7087. Min Act: -0.1700

Data Index: 4279538 (Open Web Text)

Max Activating Token Index: 150

Text #1

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.7109. Min Act: -0.1700

Data Index: 4279539 (Open Web Text)

Max Activating Token Index: 714

Text #2

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6917. Min Act: -0.1700

Data Index: 2801671 (Open Web Text)

Max Activating Token Index: 632

Text #3

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6861. Min Act: -0.1700

Data Index: 7757266 (Open Web Text)

Max Activating Token Index: 516

Text #4

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6698. Min Act: -0.1700

Data Index: 8819433 (Open Web Text)

Max Activating Token Index: 376

Text #5

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6594. Min Act: -0.1700

Data Index: 2166048 (Open Web Text)

Max Activating Token Index: 696

Text #6

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6606. Min Act: -0.1700

Data Index: 4279537 (Open Web Text)

Max Activating Token Index: 438

Text #7

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6586. Min Act: -0.1700

Data Index: 4547429 (Open Web Text)

Max Activating Token Index: 659

Text #8

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6438. Min Act: -0.1700

Data Index: 8530654 (Open Web Text)

Max Activating Token Index: 443

Text #9

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6451. Min Act: -0.1700

Data Index: 828584 (Open Web Text)

Max Activating Token Index: 479

Text #10

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6476. Min Act: -0.1700

Data Index: 5951607 (Open Web Text)

Max Activating Token Index: 863

Text #11

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6377. Min Act: -0.1700

Data Index: 3264496 (Open Web Text)

Max Activating Token Index: 155

Text #12

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6345. Min Act: -0.1700

Data Index: 5047424 (Open Web Text)

Max Activating Token Index: 643

Text #13

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6361. Min Act: -0.1700

Data Index: 1069621 (Open Web Text)

Max Activating Token Index: 892

Text #14

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6378. Min Act: -0.1700

Data Index: 5251693 (Open Web Text)

Max Activating Token Index: 681

Text #15

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6376. Min Act: -0.1700

Data Index: 1464803 (Open Web Text)

Max Activating Token Index: 735

Text #16

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6412. Min Act: -0.1700

Data Index: 962455 (Open Web Text)

Max Activating Token Index: 512

Text #17

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6263. Min Act: -0.1700

Data Index: 1612423 (Open Web Text)

Max Activating Token Index: 719

Text #18

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6322. Min Act: -0.1700

Data Index: 2216578 (Open Web Text)

Max Activating Token Index: 269

Text #19

Max Range: 1.7109. Min Range: -1.7109

Max Act: 1.6291. Min Act: -0.1700

Data Index: 116564 (Open Web Text)

Max Activating Token Index: 411