Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1391 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.7776. Min Act: -0.1700

Data Index: 2608685 (Open Web Text)

Max Activating Token Index: 719

Text #1

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.7069. Min Act: -0.1700

Data Index: 7614802 (Open Web Text)

Max Activating Token Index: 461

Text #2

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6991. Min Act: -0.1700

Data Index: 4430162 (Open Web Text)

Max Activating Token Index: 540

Text #3

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6899. Min Act: -0.1700

Data Index: 7062592 (Open Web Text)

Max Activating Token Index: 563

Text #4

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6691. Min Act: -0.1700

Data Index: 5582896 (Open Web Text)

Max Activating Token Index: 609

Text #5

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6812. Min Act: -0.1700

Data Index: 2216086 (Open Web Text)

Max Activating Token Index: 565

Text #6

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6643. Min Act: -0.1700

Data Index: 8586223 (Open Web Text)

Max Activating Token Index: 642

Text #7

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6734. Min Act: -0.1700

Data Index: 4903462 (Open Web Text)

Max Activating Token Index: 855

Text #8

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6791. Min Act: -0.1700

Data Index: 2776303 (Open Web Text)

Max Activating Token Index: 580

Text #9

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6836. Min Act: -0.1700

Data Index: 6191417 (Open Web Text)

Max Activating Token Index: 572

Text #10

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6904. Min Act: -0.1700

Data Index: 6189524 (Open Web Text)

Max Activating Token Index: 743

Text #11

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6615. Min Act: -0.1700

Data Index: 6294182 (Open Web Text)

Max Activating Token Index: 834

Text #12

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6554. Min Act: -0.1700

Data Index: 4625831 (Open Web Text)

Max Activating Token Index: 781

Text #13

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6312. Min Act: -0.1700

Data Index: 444638 (Open Web Text)

Max Activating Token Index: 715

Text #14

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6519. Min Act: -0.1700

Data Index: 275204 (Open Web Text)

Max Activating Token Index: 592

Text #15

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6419. Min Act: -0.1700

Data Index: 5399226 (Open Web Text)

Max Activating Token Index: 928

Text #16

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6480. Min Act: -0.1700

Data Index: 1025855 (Open Web Text)

Max Activating Token Index: 995

Text #17

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6393. Min Act: -0.1700

Data Index: 3298777 (Open Web Text)

Max Activating Token Index: 692

Text #18

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6621. Min Act: -0.1700

Data Index: 4233465 (Open Web Text)

Max Activating Token Index: 52

Text #19

Max Range: 3.7776. Min Range: -3.7776

Max Act: 3.6219. Min Act: -0.1700

Data Index: 737788 (Open Web Text)

Max Activating Token Index: 781