Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 122 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.9068. Min Act: -0.1700

Data Index: 5101860 (Open Web Text)

Max Activating Token Index: 81

Text #1

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.9067. Min Act: -0.1700

Data Index: 8563259 (Open Web Text)

Max Activating Token Index: 89

Text #2

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.8347. Min Act: -0.1700

Data Index: 7399925 (Open Web Text)

Max Activating Token Index: 478

Text #3

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.7553. Min Act: -0.1700

Data Index: 3645032 (Open Web Text)

Max Activating Token Index: 562

Text #4

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.7153. Min Act: -0.1700

Data Index: 5438196 (Open Web Text)

Max Activating Token Index: 37

Text #5

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.7364. Min Act: -0.1700

Data Index: 1800182 (Open Web Text)

Max Activating Token Index: 192

Text #6

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.7016. Min Act: -0.1700

Data Index: 4627764 (Open Web Text)

Max Activating Token Index: 82

Text #7

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.7043. Min Act: -0.1700

Data Index: 3254206 (Open Web Text)

Max Activating Token Index: 32

Text #8

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.7108. Min Act: -0.1700

Data Index: 7365111 (Open Web Text)

Max Activating Token Index: 1010

Text #9

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.7046. Min Act: -0.1700

Data Index: 4713788 (Open Web Text)

Max Activating Token Index: 132

Text #10

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.6869. Min Act: -0.1700

Data Index: 7237124 (Open Web Text)

Max Activating Token Index: 949

Text #11

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.6713. Min Act: -0.1700

Data Index: 3338369 (Open Web Text)

Max Activating Token Index: 23

Text #12

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.6759. Min Act: -0.1700

Data Index: 2956321 (Open Web Text)

Max Activating Token Index: 190

Text #13

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.6735. Min Act: -0.1700

Data Index: 5510273 (Open Web Text)

Max Activating Token Index: 586

Text #14

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.6529. Min Act: -0.1700

Data Index: 7218235 (Open Web Text)

Max Activating Token Index: 146

Text #15

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.6368. Min Act: -0.1700

Data Index: 3455113 (Open Web Text)

Max Activating Token Index: 935

Text #16

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.6414. Min Act: -0.1700

Data Index: 2311983 (Open Web Text)

Max Activating Token Index: 59

Text #17

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.6418. Min Act: -0.1700

Data Index: 8531834 (Open Web Text)

Max Activating Token Index: 720

Text #18

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.6144. Min Act: -0.1700

Data Index: 7020303 (Open Web Text)

Max Activating Token Index: 96

Text #19

Max Range: 3.9068. Min Range: -3.9068

Max Act: 3.6388. Min Act: -0.1700

Data Index: 5292406 (Open Web Text)

Max Activating Token Index: 763