Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 207 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.7101. Min Act: -0.1700

Data Index: 8084947 (Open Web Text)

Max Activating Token Index: 86

Text #1

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.6668. Min Act: -0.1700

Data Index: 3026866 (Open Web Text)

Max Activating Token Index: 121

Text #2

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.5798. Min Act: -0.1700

Data Index: 8759492 (Open Web Text)

Max Activating Token Index: 117

Text #3

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.5856. Min Act: -0.1700

Data Index: 978540 (Open Web Text)

Max Activating Token Index: 525

Text #4

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.5579. Min Act: -0.1700

Data Index: 1022570 (Open Web Text)

Max Activating Token Index: 531

Text #5

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.5722. Min Act: -0.1700

Data Index: 2082560 (Open Web Text)

Max Activating Token Index: 768

Text #6

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.5214. Min Act: -0.1700

Data Index: 8813130 (Open Web Text)

Max Activating Token Index: 505

Text #7

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.5349. Min Act: -0.1700

Data Index: 5241332 (Open Web Text)

Max Activating Token Index: 344

Text #8

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.5217. Min Act: -0.1700

Data Index: 8076258 (Open Web Text)

Max Activating Token Index: 442

Text #9

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.4716. Min Act: -0.1700

Data Index: 968316 (Open Web Text)

Max Activating Token Index: 987

Text #10

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.4807. Min Act: -0.1700

Data Index: 8326470 (Open Web Text)

Max Activating Token Index: 276

Text #11

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.4840. Min Act: -0.1700

Data Index: 1155657 (Open Web Text)

Max Activating Token Index: 104

Text #12

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.4643. Min Act: -0.1700

Data Index: 5004138 (Open Web Text)

Max Activating Token Index: 927

Text #13

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.4918. Min Act: -0.1700

Data Index: 2145450 (Open Web Text)

Max Activating Token Index: 761

Text #14

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.4574. Min Act: -0.1700

Data Index: 5942681 (Open Web Text)

Max Activating Token Index: 110

Text #15

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.4713. Min Act: -0.1700

Data Index: 2038176 (Open Web Text)

Max Activating Token Index: 108

Text #16

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.4468. Min Act: -0.1700

Data Index: 7776101 (Open Web Text)

Max Activating Token Index: 752

Text #17

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.4600. Min Act: -0.1700

Data Index: 5729070 (Open Web Text)

Max Activating Token Index: 345

Text #18

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.4698. Min Act: -0.1700

Data Index: 5534750 (Open Web Text)

Max Activating Token Index: 895

Text #19

Max Range: 3.7101. Min Range: -3.7101

Max Act: 3.4596. Min Act: -0.1700

Data Index: 1499549 (Open Web Text)

Max Activating Token Index: 840