Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 859 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.8383. Min Act: -0.1700

Data Index: 7088857 (Open Web Text)

Max Activating Token Index: 353

Text #1

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.7089. Min Act: -0.1700

Data Index: 6436661 (Open Web Text)

Max Activating Token Index: 458

Text #2

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.6417. Min Act: -0.1700

Data Index: 3364422 (Open Web Text)

Max Activating Token Index: 974

Text #3

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.5802. Min Act: -0.1700

Data Index: 7603821 (Open Web Text)

Max Activating Token Index: 135

Text #4

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.5614. Min Act: -0.1700

Data Index: 595667 (Open Web Text)

Max Activating Token Index: 956

Text #5

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.5435. Min Act: -0.1700

Data Index: 4269080 (Open Web Text)

Max Activating Token Index: 255

Text #6

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.5268. Min Act: -0.1700

Data Index: 6637184 (Open Web Text)

Max Activating Token Index: 426

Text #7

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.5077. Min Act: -0.1700

Data Index: 7087884 (Open Web Text)

Max Activating Token Index: 1006

Text #8

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.5128. Min Act: -0.1700

Data Index: 7282834 (Open Web Text)

Max Activating Token Index: 514

Text #9

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.4980. Min Act: -0.1700

Data Index: 7180727 (Open Web Text)

Max Activating Token Index: 890

Text #10

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.4931. Min Act: -0.1700

Data Index: 5562619 (Open Web Text)

Max Activating Token Index: 899

Text #11

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.5061. Min Act: -0.1700

Data Index: 2386116 (Open Web Text)

Max Activating Token Index: 287

Text #12

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.4167. Min Act: -0.1700

Data Index: 7369242 (Open Web Text)

Max Activating Token Index: 668

Text #13

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.4447. Min Act: -0.1700

Data Index: 3015717 (Open Web Text)

Max Activating Token Index: 955

Text #14

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.3818. Min Act: -0.1700

Data Index: 1226827 (Open Web Text)

Max Activating Token Index: 510

Text #15

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.4031. Min Act: -0.1700

Data Index: 3564135 (Open Web Text)

Max Activating Token Index: 858

Text #16

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.4758. Min Act: -0.1700

Data Index: 2027728 (Open Web Text)

Max Activating Token Index: 932

Text #17

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.4328. Min Act: -0.1700

Data Index: 6749067 (Open Web Text)

Max Activating Token Index: 848

Text #18

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.3970. Min Act: -0.1700

Data Index: 6625448 (Open Web Text)

Max Activating Token Index: 445

Text #19

Max Range: 5.8383. Min Range: -5.8383

Max Act: 5.4171. Min Act: -0.1700

Data Index: 122576 (Open Web Text)

Max Activating Token Index: 266