Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 60 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.3199. Min Act: -0.1700

Data Index: 7134772 (Open Web Text)

Max Activating Token Index: 455

Text #1

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.1980. Min Act: -0.1700

Data Index: 3798 (Open Web Text)

Max Activating Token Index: 63

Text #2

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.1483. Min Act: -0.1700

Data Index: 403623 (Open Web Text)

Max Activating Token Index: 33

Text #3

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.1104. Min Act: -0.1700

Data Index: 5753037 (Open Web Text)

Max Activating Token Index: 344

Text #4

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.1171. Min Act: -0.1700

Data Index: 4601665 (Open Web Text)

Max Activating Token Index: 85

Text #5

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0913. Min Act: -0.1700

Data Index: 8342321 (Open Web Text)

Max Activating Token Index: 455

Text #6

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0952. Min Act: -0.1700

Data Index: 7227176 (Open Web Text)

Max Activating Token Index: 83

Text #7

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0781. Min Act: -0.1700

Data Index: 4474176 (Open Web Text)

Max Activating Token Index: 31

Text #8

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0684. Min Act: -0.1700

Data Index: 876512 (Open Web Text)

Max Activating Token Index: 479

Text #9

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0567. Min Act: -0.1700

Data Index: 1667930 (Open Web Text)

Max Activating Token Index: 111

Text #10

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0469. Min Act: -0.1700

Data Index: 3706534 (Open Web Text)

Max Activating Token Index: 87

Text #11

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0515. Min Act: -0.1700

Data Index: 6609841 (Open Web Text)

Max Activating Token Index: 79

Text #12

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0399. Min Act: -0.1700

Data Index: 5312187 (Open Web Text)

Max Activating Token Index: 69

Text #13

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0257. Min Act: -0.1700

Data Index: 1660309 (Open Web Text)

Max Activating Token Index: 35

Text #14

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0122. Min Act: -0.1700

Data Index: 4474177 (Open Web Text)

Max Activating Token Index: 706

Text #15

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0030. Min Act: -0.1700

Data Index: 2458060 (Open Web Text)

Max Activating Token Index: 675

Text #16

Max Range: 3.3199. Min Range: -3.3199

Max Act: 2.9953. Min Act: -0.1700

Data Index: 2245146 (Open Web Text)

Max Activating Token Index: 910

Text #17

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0034. Min Act: -0.1700

Data Index: 5547949 (Open Web Text)

Max Activating Token Index: 25

Text #18

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0090. Min Act: -0.1700

Data Index: 4853419 (Open Web Text)

Max Activating Token Index: 48

Text #19

Max Range: 3.3199. Min Range: -3.3199

Max Act: 3.0109. Min Act: -0.1700

Data Index: 3706533 (Open Web Text)

Max Activating Token Index: 513