Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 657 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6938. Min Act: -0.1700

Data Index: 7164258 (Open Web Text)

Max Activating Token Index: 683

Text #1

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6463. Min Act: -0.1700

Data Index: 46579 (Open Web Text)

Max Activating Token Index: 676

Text #2

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6297. Min Act: -0.1700

Data Index: 1125824 (Open Web Text)

Max Activating Token Index: 724

Text #3

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6258. Min Act: -0.1700

Data Index: 357479 (Open Web Text)

Max Activating Token Index: 674

Text #4

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6247. Min Act: -0.1700

Data Index: 5726222 (Open Web Text)

Max Activating Token Index: 549

Text #5

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6087. Min Act: -0.1700

Data Index: 7862995 (Open Web Text)

Max Activating Token Index: 1004

Text #6

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6149. Min Act: -0.1700

Data Index: 7303200 (Open Web Text)

Max Activating Token Index: 789

Text #7

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6153. Min Act: -0.1700

Data Index: 3517486 (Open Web Text)

Max Activating Token Index: 175

Text #8

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6147. Min Act: -0.1700

Data Index: 4393812 (Open Web Text)

Max Activating Token Index: 971

Text #9

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6081. Min Act: -0.1700

Data Index: 8093269 (Open Web Text)

Max Activating Token Index: 531

Text #10

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6053. Min Act: -0.1700

Data Index: 6152506 (Open Web Text)

Max Activating Token Index: 47

Text #11

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6113. Min Act: -0.1700

Data Index: 5084879 (Open Web Text)

Max Activating Token Index: 320

Text #12

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.5960. Min Act: -0.1700

Data Index: 5559202 (Open Web Text)

Max Activating Token Index: 1002

Text #13

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.5970. Min Act: -0.1700

Data Index: 7769316 (Open Web Text)

Max Activating Token Index: 785

Text #14

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.5996. Min Act: -0.1700

Data Index: 8365002 (Open Web Text)

Max Activating Token Index: 576

Text #15

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.5954. Min Act: -0.1700

Data Index: 3683400 (Open Web Text)

Max Activating Token Index: 712

Text #16

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6061. Min Act: -0.1700

Data Index: 8752765 (Open Web Text)

Max Activating Token Index: 247

Text #17

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.5943. Min Act: -0.1700

Data Index: 4020913 (Open Web Text)

Max Activating Token Index: 398

Text #18

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.6070. Min Act: -0.1700

Data Index: 5328538 (Open Web Text)

Max Activating Token Index: 375

Text #19

Max Range: 1.6938. Min Range: -1.6938

Max Act: 1.5864. Min Act: -0.1700

Data Index: 5253871 (Open Web Text)

Max Activating Token Index: 581