Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1279 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.1922. Min Act: -0.1700

Data Index: 5053072 (Open Web Text)

Max Activating Token Index: 94

Text #1

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.1568. Min Act: -0.1700

Data Index: 318167 (Open Web Text)

Max Activating Token Index: 93

Text #2

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.1298. Min Act: -0.1700

Data Index: 4494738 (Open Web Text)

Max Activating Token Index: 96

Text #3

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0971. Min Act: -0.1700

Data Index: 5657015 (Open Web Text)

Max Activating Token Index: 9

Text #4

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.1060. Min Act: -0.1700

Data Index: 2441150 (Open Web Text)

Max Activating Token Index: 9

Text #5

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0900. Min Act: -0.1700

Data Index: 4836740 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0701. Min Act: -0.1700

Data Index: 1274904 (Open Web Text)

Max Activating Token Index: 2

Text #7

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0701. Min Act: -0.1700

Data Index: 8004647 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0701. Min Act: -0.1700

Data Index: 6699961 (Open Web Text)

Max Activating Token Index: 2

Text #9

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0919. Min Act: -0.1700

Data Index: 6759010 (Open Web Text)

Max Activating Token Index: 41

Text #10

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0701. Min Act: -0.1700

Data Index: 6663574 (Open Web Text)

Max Activating Token Index: 2

Text #11

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0658. Min Act: -0.1700

Data Index: 7335576 (Open Web Text)

Max Activating Token Index: 10

Text #12

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0701. Min Act: -0.1700

Data Index: 2679834 (Open Web Text)

Max Activating Token Index: 2

Text #13

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0532. Min Act: -0.1700

Data Index: 7960597 (Open Web Text)

Max Activating Token Index: 8

Text #14

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0379. Min Act: -0.1700

Data Index: 4395354 (Open Web Text)

Max Activating Token Index: 850

Text #15

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0562. Min Act: -0.1700

Data Index: 5323272 (Open Web Text)

Max Activating Token Index: 13

Text #16

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0625. Min Act: -0.1700

Data Index: 7829234 (Open Web Text)

Max Activating Token Index: 11

Text #17

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0638. Min Act: -0.1700

Data Index: 2370064 (Open Web Text)

Max Activating Token Index: 9

Text #18

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0521. Min Act: -0.1700

Data Index: 7910709 (Open Web Text)

Max Activating Token Index: 185

Text #19

Max Range: 3.1922. Min Range: -3.1922

Max Act: 3.0615. Min Act: -0.1700

Data Index: 2427004 (Open Web Text)

Max Activating Token Index: 7