Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4058 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0331. Min Range: -1.0331

Max Act: 1.0331. Min Act: -0.0000

Data Index: 157538 (The Pile)

Max Activating Token Index: 733

Text #1

Max Range: 1.0331. Min Range: -1.0331

Max Act: 1.0139. Min Act: -0.0000

Data Index: 688726 (The Pile)

Max Activating Token Index: 634

Text #2

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.9223. Min Act: -0.0000

Data Index: 892550 (The Pile)

Max Activating Token Index: 752

Text #3

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.9381. Min Act: -0.0000

Data Index: 983703 (The Pile)

Max Activating Token Index: 87

Text #4

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.9275. Min Act: -0.0000

Data Index: 479687 (The Pile)

Max Activating Token Index: 30

Text #5

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.9594. Min Act: -0.0000

Data Index: 333460 (The Pile)

Max Activating Token Index: 459

Text #6

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.9139. Min Act: -0.0000

Data Index: 1960418 (The Pile)

Max Activating Token Index: 449

Text #7

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.8944. Min Act: -0.0000

Data Index: 693942 (The Pile)

Max Activating Token Index: 991

Text #8

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.8849. Min Act: -0.0000

Data Index: 1468750 (The Pile)

Max Activating Token Index: 496

Text #9

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.8788. Min Act: -0.0000

Data Index: 927716 (The Pile)

Max Activating Token Index: 203

Text #10

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.8756. Min Act: -0.0000

Data Index: 1681829 (The Pile)

Max Activating Token Index: 869

Text #11

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.8668. Min Act: -0.0000

Data Index: 1464081 (The Pile)

Max Activating Token Index: 600

Text #12

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.8593. Min Act: -0.0000

Data Index: 1165236 (The Pile)

Max Activating Token Index: 860

Text #13

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.8357. Min Act: -0.0000

Data Index: 1542877 (The Pile)

Max Activating Token Index: 493

Text #14

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.8873. Min Act: -0.0000

Data Index: 1182084 (The Pile)

Max Activating Token Index: 891

Text #15

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.8485. Min Act: -0.0000

Data Index: 1319224 (The Pile)

Max Activating Token Index: 363

Text #16

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.8341. Min Act: -0.0000

Data Index: 1164752 (The Pile)

Max Activating Token Index: 478

Text #17

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.7789. Min Act: -0.0000

Data Index: 761747 (The Pile)

Max Activating Token Index: 421

Text #18

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.8452. Min Act: -0.0000

Data Index: 1236107 (The Pile)

Max Activating Token Index: 1015

Text #19

Max Range: 1.0331. Min Range: -1.0331

Max Act: 0.7684. Min Act: -0.0000

Data Index: 791772 (The Pile)

Max Activating Token Index: 631