Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2751 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.7894. Min Act: -0.1700

Data Index: 796129 (Open Web Text)

Max Activating Token Index: 81

Text #1

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.7922. Min Act: -0.1700

Data Index: 1449554 (Open Web Text)

Max Activating Token Index: 202

Text #2

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.7965. Min Act: -0.1700

Data Index: 5514445 (Open Web Text)

Max Activating Token Index: 308

Text #3

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.7658. Min Act: -0.1700

Data Index: 1592859 (Open Web Text)

Max Activating Token Index: 499

Text #4

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.7393. Min Act: -0.1700

Data Index: 3448848 (Open Web Text)

Max Activating Token Index: 680

Text #5

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.7843. Min Act: -0.1700

Data Index: 506478 (Open Web Text)

Max Activating Token Index: 93

Text #6

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.6832. Min Act: -0.1700

Data Index: 8819851 (Open Web Text)

Max Activating Token Index: 461

Text #7

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.7023. Min Act: -0.1700

Data Index: 8004605 (Open Web Text)

Max Activating Token Index: 51

Text #8

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.7267. Min Act: -0.1700

Data Index: 7899743 (Open Web Text)

Max Activating Token Index: 595

Text #9

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.7038. Min Act: -0.1700

Data Index: 7883527 (Open Web Text)

Max Activating Token Index: 989

Text #10

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.7124. Min Act: -0.1700

Data Index: 1407466 (Open Web Text)

Max Activating Token Index: 270

Text #11

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.6635. Min Act: -0.1700

Data Index: 5554477 (Open Web Text)

Max Activating Token Index: 366

Text #12

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.6874. Min Act: -0.1700

Data Index: 7547574 (Open Web Text)

Max Activating Token Index: 800

Text #13

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.6896. Min Act: -0.1700

Data Index: 3643705 (Open Web Text)

Max Activating Token Index: 532

Text #14

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.6424. Min Act: -0.1700

Data Index: 5990109 (Open Web Text)

Max Activating Token Index: 203

Text #15

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.6303. Min Act: -0.1700

Data Index: 5668226 (Open Web Text)

Max Activating Token Index: 520

Text #16

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.6438. Min Act: -0.1700

Data Index: 7930449 (Open Web Text)

Max Activating Token Index: 82

Text #17

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.6436. Min Act: -0.1700

Data Index: 6862890 (Open Web Text)

Max Activating Token Index: 44

Text #18

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.6598. Min Act: -0.1700

Data Index: 1399579 (Open Web Text)

Max Activating Token Index: 865

Text #19

Max Range: 3.7965. Min Range: -3.7965

Max Act: 3.6225. Min Act: -0.1700

Data Index: 1265087 (Open Web Text)

Max Activating Token Index: 644