Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2948 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0028. Min Range: -4.0028

Max Act: 4.0028. Min Act: -0.1700

Data Index: 2828328 (Open Web Text)

Max Activating Token Index: 429

Text #1

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.8898. Min Act: -0.1700

Data Index: 2940882 (Open Web Text)

Max Activating Token Index: 128

Text #2

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.8702. Min Act: -0.1700

Data Index: 4217979 (Open Web Text)

Max Activating Token Index: 492

Text #3

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.8592. Min Act: -0.1700

Data Index: 3310184 (Open Web Text)

Max Activating Token Index: 637

Text #4

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.8158. Min Act: -0.1700

Data Index: 5288921 (Open Web Text)

Max Activating Token Index: 567

Text #5

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.8274. Min Act: -0.1700

Data Index: 392805 (Open Web Text)

Max Activating Token Index: 213

Text #6

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.7856. Min Act: -0.1700

Data Index: 1729125 (Open Web Text)

Max Activating Token Index: 182

Text #7

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.8050. Min Act: -0.1700

Data Index: 3423587 (Open Web Text)

Max Activating Token Index: 637

Text #8

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.8104. Min Act: -0.1700

Data Index: 6766134 (Open Web Text)

Max Activating Token Index: 956

Text #9

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.7730. Min Act: -0.1700

Data Index: 8193129 (Open Web Text)

Max Activating Token Index: 596

Text #10

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.7675. Min Act: -0.1700

Data Index: 1966503 (Open Web Text)

Max Activating Token Index: 946

Text #11

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.7586. Min Act: -0.1700

Data Index: 1345976 (Open Web Text)

Max Activating Token Index: 580

Text #12

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.6880. Min Act: -0.1700

Data Index: 8515092 (Open Web Text)

Max Activating Token Index: 59

Text #13

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.6863. Min Act: -0.1700

Data Index: 4397734 (Open Web Text)

Max Activating Token Index: 333

Text #14

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.7003. Min Act: -0.1700

Data Index: 7049626 (Open Web Text)

Max Activating Token Index: 578

Text #15

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.7360. Min Act: -0.1700

Data Index: 7656566 (Open Web Text)

Max Activating Token Index: 498

Text #16

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.7159. Min Act: -0.1700

Data Index: 6482569 (Open Web Text)

Max Activating Token Index: 311

Text #17

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.6815. Min Act: -0.1700

Data Index: 5014755 (Open Web Text)

Max Activating Token Index: 280

Text #18

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.6446. Min Act: -0.1700

Data Index: 5829265 (Open Web Text)

Max Activating Token Index: 790

Text #19

Max Range: 4.0028. Min Range: -4.0028

Max Act: 3.6645. Min Act: -0.1700

Data Index: 6257184 (Open Web Text)

Max Activating Token Index: 619