Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3751 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.8682. Min Act: -0.0000

Data Index: 763996 (The Pile)

Max Activating Token Index: 987

Text #1

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.7694. Min Act: -0.0000

Data Index: 1679762 (The Pile)

Max Activating Token Index: 995

Text #2

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6832. Min Act: -0.0000

Data Index: 1556348 (The Pile)

Max Activating Token Index: 988

Text #3

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.7005. Min Act: -0.0000

Data Index: 752764 (The Pile)

Max Activating Token Index: 994

Text #4

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6927. Min Act: -0.0000

Data Index: 1956472 (The Pile)

Max Activating Token Index: 383

Text #5

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6877. Min Act: -0.0000

Data Index: 1886370 (The Pile)

Max Activating Token Index: 709

Text #6

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.7100. Min Act: -0.0000

Data Index: 109360 (The Pile)

Max Activating Token Index: 1006

Text #7

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6539. Min Act: -0.0000

Data Index: 1497990 (The Pile)

Max Activating Token Index: 697

Text #8

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6757. Min Act: -0.0000

Data Index: 348906 (The Pile)

Max Activating Token Index: 742

Text #9

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6878. Min Act: -0.0000

Data Index: 1845194 (The Pile)

Max Activating Token Index: 775

Text #10

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6681. Min Act: -0.0000

Data Index: 149486 (The Pile)

Max Activating Token Index: 661

Text #11

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6600. Min Act: 0.0000

Data Index: 777415 (The Pile)

Max Activating Token Index: 60

Text #12

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6604. Min Act: -0.0000

Data Index: 771617 (The Pile)

Max Activating Token Index: 184

Text #13

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6358. Min Act: -0.0000

Data Index: 77835 (The Pile)

Max Activating Token Index: 538

Text #14

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6543. Min Act: -0.0000

Data Index: 808376 (The Pile)

Max Activating Token Index: 296

Text #15

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6497. Min Act: -0.0000

Data Index: 423112 (The Pile)

Max Activating Token Index: 232

Text #16

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6321. Min Act: -0.0000

Data Index: 1443420 (The Pile)

Max Activating Token Index: 489

Text #17

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6465. Min Act: -0.0000

Data Index: 1161527 (The Pile)

Max Activating Token Index: 315

Text #18

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6467. Min Act: -0.0000

Data Index: 604863 (The Pile)

Max Activating Token Index: 277

Text #19

Max Range: 0.8682. Min Range: -0.8682

Max Act: 0.6464. Min Act: -0.0000

Data Index: 1847063 (The Pile)

Max Activating Token Index: 541