Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2720 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6686. Min Act: -0.1700

Data Index: 1420566 (Open Web Text)

Max Activating Token Index: 55

Text #1

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6520. Min Act: -0.1700

Data Index: 5615073 (Open Web Text)

Max Activating Token Index: 688

Text #2

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6462. Min Act: -0.1698

Data Index: 86450 (Open Web Text)

Max Activating Token Index: 55

Text #3

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6611. Min Act: -0.1700

Data Index: 5823466 (Open Web Text)

Max Activating Token Index: 69

Text #4

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6336. Min Act: -0.1700

Data Index: 3660768 (Open Web Text)

Max Activating Token Index: 184

Text #5

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6480. Min Act: -0.1700

Data Index: 2166826 (Open Web Text)

Max Activating Token Index: 218

Text #6

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6462. Min Act: -0.1691

Data Index: 2226309 (Open Web Text)

Max Activating Token Index: 503

Text #7

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6307. Min Act: -0.1699

Data Index: 428486 (Open Web Text)

Max Activating Token Index: 123

Text #8

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6267. Min Act: -0.1700

Data Index: 1100948 (Open Web Text)

Max Activating Token Index: 345

Text #9

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6158. Min Act: -0.1699

Data Index: 834437 (Open Web Text)

Max Activating Token Index: 536

Text #10

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6290. Min Act: -0.1700

Data Index: 7565581 (Open Web Text)

Max Activating Token Index: 95

Text #11

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6287. Min Act: -0.1700

Data Index: 1665209 (Open Web Text)

Max Activating Token Index: 411

Text #12

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6384. Min Act: -0.1700

Data Index: 6190433 (Open Web Text)

Max Activating Token Index: 712

Text #13

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6215. Min Act: -0.1700

Data Index: 1697690 (Open Web Text)

Max Activating Token Index: 809

Text #14

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6307. Min Act: -0.1700

Data Index: 7858711 (Open Web Text)

Max Activating Token Index: 686

Text #15

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6263. Min Act: -0.1700

Data Index: 923379 (Open Web Text)

Max Activating Token Index: 954

Text #16

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6140. Min Act: -0.1700

Data Index: 2042233 (Open Web Text)

Max Activating Token Index: 771

Text #17

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6147. Min Act: -0.1700

Data Index: 6941348 (Open Web Text)

Max Activating Token Index: 188

Text #18

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6149. Min Act: -0.1699

Data Index: 1029449 (Open Web Text)

Max Activating Token Index: 613

Text #19

Max Range: 1.6686. Min Range: -1.6686

Max Act: 1.6159. Min Act: -0.1700

Data Index: 5855224 (Open Web Text)

Max Activating Token Index: 573