Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2204 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.7882. Min Act: -0.1699

Data Index: 3054669 (Open Web Text)

Max Activating Token Index: 702

Text #1

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.7620. Min Act: -0.1699

Data Index: 2972078 (Open Web Text)

Max Activating Token Index: 820

Text #2

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.7637. Min Act: -0.1694

Data Index: 3604694 (Open Web Text)

Max Activating Token Index: 705

Text #3

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.7333. Min Act: -0.1700

Data Index: 6310514 (Open Web Text)

Max Activating Token Index: 597

Text #4

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6954. Min Act: -0.1700

Data Index: 3356977 (Open Web Text)

Max Activating Token Index: 521

Text #5

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6969. Min Act: -0.1700

Data Index: 787654 (Open Web Text)

Max Activating Token Index: 158

Text #6

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6863. Min Act: -0.1700

Data Index: 2531646 (Open Web Text)

Max Activating Token Index: 36

Text #7

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6789. Min Act: -0.1699

Data Index: 8251266 (Open Web Text)

Max Activating Token Index: 52

Text #8

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6919. Min Act: -0.1699

Data Index: 8100293 (Open Web Text)

Max Activating Token Index: 779

Text #9

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6807. Min Act: -0.1700

Data Index: 3862337 (Open Web Text)

Max Activating Token Index: 844

Text #10

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6716. Min Act: -0.1689

Data Index: 4026676 (Open Web Text)

Max Activating Token Index: 420

Text #11

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6609. Min Act: -0.1700

Data Index: 644664 (Open Web Text)

Max Activating Token Index: 640

Text #12

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6740. Min Act: -0.1696

Data Index: 2701876 (Open Web Text)

Max Activating Token Index: 638

Text #13

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6580. Min Act: -0.1690

Data Index: 4741078 (Open Web Text)

Max Activating Token Index: 853

Text #14

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6683. Min Act: -0.1699

Data Index: 2539811 (Open Web Text)

Max Activating Token Index: 421

Text #15

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6558. Min Act: -0.1696

Data Index: 5267042 (Open Web Text)

Max Activating Token Index: 438

Text #16

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6729. Min Act: -0.1695

Data Index: 779915 (Open Web Text)

Max Activating Token Index: 560

Text #17

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6458. Min Act: -0.1700

Data Index: 1034818 (Open Web Text)

Max Activating Token Index: 336

Text #18

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6514. Min Act: -0.1700

Data Index: 3922684 (Open Web Text)

Max Activating Token Index: 52

Text #19

Max Range: 1.7882. Min Range: -1.7882

Max Act: 1.6533. Min Act: -0.1700

Data Index: 3731620 (Open Web Text)

Max Activating Token Index: 558