Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 709 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.9262. Min Act: -0.1700

Data Index: 1416684 (Open Web Text)

Max Activating Token Index: 515

Text #1

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.9508. Min Act: -0.1700

Data Index: 909394 (Open Web Text)

Max Activating Token Index: 248

Text #2

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.8316. Min Act: -0.1700

Data Index: 7768295 (Open Web Text)

Max Activating Token Index: 361

Text #3

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.7844. Min Act: -0.1700

Data Index: 388474 (Open Web Text)

Max Activating Token Index: 68

Text #4

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.7992. Min Act: -0.1700

Data Index: 3620236 (Open Web Text)

Max Activating Token Index: 925

Text #5

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.7632. Min Act: -0.1700

Data Index: 8645833 (Open Web Text)

Max Activating Token Index: 959

Text #6

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.7534. Min Act: -0.1700

Data Index: 2108129 (Open Web Text)

Max Activating Token Index: 319

Text #7

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.7565. Min Act: -0.1700

Data Index: 2507109 (Open Web Text)

Max Activating Token Index: 46

Text #8

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.7574. Min Act: -0.1700

Data Index: 306172 (Open Web Text)

Max Activating Token Index: 684

Text #9

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.7495. Min Act: -0.1700

Data Index: 1949828 (Open Web Text)

Max Activating Token Index: 616

Text #10

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.7094. Min Act: -0.1700

Data Index: 2721253 (Open Web Text)

Max Activating Token Index: 177

Text #11

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.7684. Min Act: -0.1700

Data Index: 325021 (Open Web Text)

Max Activating Token Index: 847

Text #12

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.7422. Min Act: -0.1700

Data Index: 5499678 (Open Web Text)

Max Activating Token Index: 668

Text #13

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.6966. Min Act: -0.1700

Data Index: 2761579 (Open Web Text)

Max Activating Token Index: 148

Text #14

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.7289. Min Act: -0.1700

Data Index: 2971985 (Open Web Text)

Max Activating Token Index: 822

Text #15

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.6877. Min Act: -0.1700

Data Index: 4897202 (Open Web Text)

Max Activating Token Index: 146

Text #16

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.7054. Min Act: -0.1700

Data Index: 3754493 (Open Web Text)

Max Activating Token Index: 446

Text #17

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.6831. Min Act: -0.1700

Data Index: 628255 (Open Web Text)

Max Activating Token Index: 50

Text #18

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.6991. Min Act: -0.1700

Data Index: 3327530 (Open Web Text)

Max Activating Token Index: 992

Text #19

Max Range: 4.9508. Min Range: -4.9508

Max Act: 4.6897. Min Act: -0.1700

Data Index: 664502 (Open Web Text)

Max Activating Token Index: 653