Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 882 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.5117. Min Act: -0.1700

Data Index: 2306937 (Open Web Text)

Max Activating Token Index: 870

Text #1

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.4957. Min Act: -0.1700

Data Index: 8701982 (Open Web Text)

Max Activating Token Index: 441

Text #2

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.3953. Min Act: -0.1700

Data Index: 850722 (Open Web Text)

Max Activating Token Index: 836

Text #3

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.4318. Min Act: -0.1700

Data Index: 2769671 (Open Web Text)

Max Activating Token Index: 840

Text #4

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.4045. Min Act: -0.1700

Data Index: 6059336 (Open Web Text)

Max Activating Token Index: 970

Text #5

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.4108. Min Act: -0.1700

Data Index: 4733828 (Open Web Text)

Max Activating Token Index: 891

Text #6

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.3847. Min Act: -0.1700

Data Index: 1703841 (Open Web Text)

Max Activating Token Index: 835

Text #7

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.3495. Min Act: -0.1700

Data Index: 234300 (Open Web Text)

Max Activating Token Index: 843

Text #8

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.3245. Min Act: -0.1700

Data Index: 7868151 (Open Web Text)

Max Activating Token Index: 349

Text #9

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.2925. Min Act: -0.1700

Data Index: 8608810 (Open Web Text)

Max Activating Token Index: 997

Text #10

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.3011. Min Act: -0.1700

Data Index: 3097518 (Open Web Text)

Max Activating Token Index: 402

Text #11

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.3208. Min Act: -0.1700

Data Index: 4041231 (Open Web Text)

Max Activating Token Index: 619

Text #12

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.3272. Min Act: -0.1700

Data Index: 1718482 (Open Web Text)

Max Activating Token Index: 863

Text #13

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.3153. Min Act: -0.1700

Data Index: 1327944 (Open Web Text)

Max Activating Token Index: 982

Text #14

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.3710. Min Act: -0.1700

Data Index: 3044643 (Open Web Text)

Max Activating Token Index: 254

Text #15

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.2774. Min Act: -0.1700

Data Index: 2871277 (Open Web Text)

Max Activating Token Index: 461

Text #16

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.2931. Min Act: -0.1700

Data Index: 4448642 (Open Web Text)

Max Activating Token Index: 957

Text #17

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.2248. Min Act: -0.1700

Data Index: 45392 (Open Web Text)

Max Activating Token Index: 147

Text #18

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.2689. Min Act: -0.1700

Data Index: 1176621 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 4.5117. Min Range: -4.5117

Max Act: 4.2655. Min Act: -0.1700

Data Index: 4476697 (Open Web Text)

Max Activating Token Index: 4