Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2242 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5955. Min Act: -0.1700

Data Index: 4356372 (Open Web Text)

Max Activating Token Index: 117

Text #1

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5923. Min Act: -0.1700

Data Index: 467080 (Open Web Text)

Max Activating Token Index: 558

Text #2

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.6047. Min Act: -0.1700

Data Index: 3536067 (Open Web Text)

Max Activating Token Index: 82

Text #3

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.6120. Min Act: -0.1700

Data Index: 3051008 (Open Web Text)

Max Activating Token Index: 198

Text #4

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5958. Min Act: -0.1700

Data Index: 1735668 (Open Web Text)

Max Activating Token Index: 594

Text #5

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5699. Min Act: -0.1700

Data Index: 4392696 (Open Web Text)

Max Activating Token Index: 345

Text #6

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5587. Min Act: -0.1700

Data Index: 6383949 (Open Web Text)

Max Activating Token Index: 90

Text #7

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5654. Min Act: -0.1700

Data Index: 7770260 (Open Web Text)

Max Activating Token Index: 245

Text #8

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5594. Min Act: -0.1700

Data Index: 6858387 (Open Web Text)

Max Activating Token Index: 121

Text #9

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5532. Min Act: -0.1700

Data Index: 2222898 (Open Web Text)

Max Activating Token Index: 962

Text #10

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5216. Min Act: -0.1700

Data Index: 5810059 (Open Web Text)

Max Activating Token Index: 170

Text #11

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5290. Min Act: -0.1700

Data Index: 2932453 (Open Web Text)

Max Activating Token Index: 304

Text #12

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5295. Min Act: -0.1700

Data Index: 7420696 (Open Web Text)

Max Activating Token Index: 561

Text #13

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5233. Min Act: -0.1700

Data Index: 580943 (Open Web Text)

Max Activating Token Index: 538

Text #14

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5175. Min Act: -0.1700

Data Index: 5826660 (Open Web Text)

Max Activating Token Index: 83

Text #15

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5291. Min Act: -0.1700

Data Index: 1661411 (Open Web Text)

Max Activating Token Index: 159

Text #16

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5333. Min Act: -0.1700

Data Index: 6384438 (Open Web Text)

Max Activating Token Index: 214

Text #17

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5038. Min Act: -0.1700

Data Index: 4487796 (Open Web Text)

Max Activating Token Index: 158

Text #18

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5070. Min Act: -0.1700

Data Index: 6477905 (Open Web Text)

Max Activating Token Index: 991

Text #19

Max Range: 4.6120. Min Range: -4.6120

Max Act: 4.5093. Min Act: -0.1700

Data Index: 2164656 (Open Web Text)

Max Activating Token Index: 531