Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3622 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.1949. Min Act: -0.0000

Data Index: 1528506 (The Pile)

Max Activating Token Index: 1000

Text #1

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.1762. Min Act: -0.0000

Data Index: 778460 (The Pile)

Max Activating Token Index: 385

Text #2

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.0778. Min Act: -0.0000

Data Index: 929969 (The Pile)

Max Activating Token Index: 260

Text #3

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.0898. Min Act: -0.0000

Data Index: 1526361 (The Pile)

Max Activating Token Index: 343

Text #4

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.0788. Min Act: -0.0000

Data Index: 693648 (The Pile)

Max Activating Token Index: 929

Text #5

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.0631. Min Act: -0.0000

Data Index: 1113021 (The Pile)

Max Activating Token Index: 448

Text #6

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.0222. Min Act: -0.0000

Data Index: 1013981 (The Pile)

Max Activating Token Index: 841

Text #7

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.0556. Min Act: -0.0000

Data Index: 1768612 (The Pile)

Max Activating Token Index: 670

Text #8

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.0037. Min Act: -0.0000

Data Index: 550405 (The Pile)

Max Activating Token Index: 233

Text #9

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.0322. Min Act: -0.0000

Data Index: 160834 (The Pile)

Max Activating Token Index: 233

Text #10

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.0049. Min Act: -0.0000

Data Index: 794515 (The Pile)

Max Activating Token Index: 226

Text #11

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.0021. Min Act: -0.0000

Data Index: 886575 (The Pile)

Max Activating Token Index: 526

Text #12

Max Range: 1.1949. Min Range: -1.1949

Max Act: 0.9492. Min Act: -0.0000

Data Index: 1026618 (The Pile)

Max Activating Token Index: 844

Text #13

Max Range: 1.1949. Min Range: -1.1949

Max Act: 1.0273. Min Act: -0.0000

Data Index: 606340 (The Pile)

Max Activating Token Index: 125

Text #14

Max Range: 1.1949. Min Range: -1.1949

Max Act: 0.9740. Min Act: -0.0000

Data Index: 411976 (The Pile)

Max Activating Token Index: 70

Text #15

Max Range: 1.1949. Min Range: -1.1949

Max Act: 0.9703. Min Act: -0.0000

Data Index: 1360463 (The Pile)

Max Activating Token Index: 598

Text #16

Max Range: 1.1949. Min Range: -1.1949

Max Act: 0.9975. Min Act: -0.0000

Data Index: 1158827 (The Pile)

Max Activating Token Index: 779

Text #17

Max Range: 1.1949. Min Range: -1.1949

Max Act: 0.9679. Min Act: -0.0000

Data Index: 1130208 (The Pile)

Max Activating Token Index: 53

Text #18

Max Range: 1.1949. Min Range: -1.1949

Max Act: 0.9329. Min Act: -0.0000

Data Index: 1864120 (The Pile)

Max Activating Token Index: 533

Text #19

Max Range: 1.1949. Min Range: -1.1949

Max Act: 0.9001. Min Act: -0.0000

Data Index: 903193 (The Pile)

Max Activating Token Index: 273