Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 123 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.6938. Min Act: -0.1700

Data Index: 3618910 (Open Web Text)

Max Activating Token Index: 480

Text #1

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.6994. Min Act: -0.1700

Data Index: 953487 (Open Web Text)

Max Activating Token Index: 688

Text #2

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.7255. Min Act: -0.1700

Data Index: 1732833 (Open Web Text)

Max Activating Token Index: 772

Text #3

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.6797. Min Act: -0.1700

Data Index: 3539893 (Open Web Text)

Max Activating Token Index: 442

Text #4

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.6687. Min Act: -0.1700

Data Index: 1232978 (Open Web Text)

Max Activating Token Index: 217

Text #5

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.6758. Min Act: -0.1700

Data Index: 4642126 (Open Web Text)

Max Activating Token Index: 387

Text #6

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.6421. Min Act: -0.1700

Data Index: 4494836 (Open Web Text)

Max Activating Token Index: 937

Text #7

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.6781. Min Act: -0.1700

Data Index: 1894648 (Open Web Text)

Max Activating Token Index: 374

Text #8

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.6517. Min Act: -0.1700

Data Index: 2706290 (Open Web Text)

Max Activating Token Index: 891

Text #9

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.6667. Min Act: -0.1700

Data Index: 4637236 (Open Web Text)

Max Activating Token Index: 775

Text #10

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.6104. Min Act: -0.1700

Data Index: 450692 (Open Web Text)

Max Activating Token Index: 885

Text #11

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.5871. Min Act: -0.1700

Data Index: 7692613 (Open Web Text)

Max Activating Token Index: 619

Text #12

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.5771. Min Act: -0.1700

Data Index: 6398094 (Open Web Text)

Max Activating Token Index: 782

Text #13

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.6066. Min Act: -0.1700

Data Index: 6999458 (Open Web Text)

Max Activating Token Index: 648

Text #14

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.5868. Min Act: -0.1700

Data Index: 3470159 (Open Web Text)

Max Activating Token Index: 479

Text #15

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.5933. Min Act: -0.1700

Data Index: 2560754 (Open Web Text)

Max Activating Token Index: 1020

Text #16

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.5943. Min Act: -0.1700

Data Index: 816979 (Open Web Text)

Max Activating Token Index: 745

Text #17

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.5833. Min Act: -0.1700

Data Index: 1813206 (Open Web Text)

Max Activating Token Index: 820

Text #18

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.5824. Min Act: -0.1700

Data Index: 1672199 (Open Web Text)

Max Activating Token Index: 556

Text #19

Max Range: 4.7255. Min Range: -4.7255

Max Act: 4.5597. Min Act: -0.1700

Data Index: 3516684 (Open Web Text)

Max Activating Token Index: 803