Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2970 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0483. Min Range: -4.0483

Max Act: 4.0483. Min Act: -0.1700

Data Index: 7442103 (Open Web Text)

Max Activating Token Index: 941

Text #1

Max Range: 4.0483. Min Range: -4.0483

Max Act: 4.0463. Min Act: -0.1700

Data Index: 4350538 (Open Web Text)

Max Activating Token Index: 430

Text #2

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.9668. Min Act: -0.1700

Data Index: 82757 (Open Web Text)

Max Activating Token Index: 684

Text #3

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.9435. Min Act: -0.1700

Data Index: 610445 (Open Web Text)

Max Activating Token Index: 85

Text #4

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.9543. Min Act: -0.1700

Data Index: 3339553 (Open Web Text)

Max Activating Token Index: 137

Text #5

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.9360. Min Act: -0.1700

Data Index: 112939 (Open Web Text)

Max Activating Token Index: 872

Text #6

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.9471. Min Act: -0.1700

Data Index: 7240491 (Open Web Text)

Max Activating Token Index: 500

Text #7

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.8862. Min Act: -0.1700

Data Index: 4746299 (Open Web Text)

Max Activating Token Index: 140

Text #8

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.9156. Min Act: -0.1700

Data Index: 1509066 (Open Web Text)

Max Activating Token Index: 214

Text #9

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.9039. Min Act: -0.1700

Data Index: 285662 (Open Web Text)

Max Activating Token Index: 567

Text #10

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.8939. Min Act: -0.1700

Data Index: 4322187 (Open Web Text)

Max Activating Token Index: 234

Text #11

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.8552. Min Act: -0.1700

Data Index: 2883284 (Open Web Text)

Max Activating Token Index: 137

Text #12

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.8861. Min Act: -0.1700

Data Index: 4153922 (Open Web Text)

Max Activating Token Index: 559

Text #13

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.9064. Min Act: -0.1700

Data Index: 5510318 (Open Web Text)

Max Activating Token Index: 822

Text #14

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.8678. Min Act: -0.1700

Data Index: 7168281 (Open Web Text)

Max Activating Token Index: 615

Text #15

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.8987. Min Act: -0.1700

Data Index: 6973197 (Open Web Text)

Max Activating Token Index: 302

Text #16

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.8856. Min Act: -0.1700

Data Index: 6257239 (Open Web Text)

Max Activating Token Index: 736

Text #17

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.8477. Min Act: -0.1700

Data Index: 5774564 (Open Web Text)

Max Activating Token Index: 182

Text #18

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.8653. Min Act: -0.1700

Data Index: 1509414 (Open Web Text)

Max Activating Token Index: 329

Text #19

Max Range: 4.0483. Min Range: -4.0483

Max Act: 3.8641. Min Act: -0.1700

Data Index: 2162767 (Open Web Text)

Max Activating Token Index: 206