Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4609 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0673. Min Act: -0.0000

Data Index: 1235694 (The Pile)

Max Activating Token Index: 75

Text #1

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0645. Min Act: -0.0000

Data Index: 1335976 (The Pile)

Max Activating Token Index: 987

Text #2

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0635. Min Act: -0.0000

Data Index: 372440 (The Pile)

Max Activating Token Index: 86

Text #3

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0625. Min Act: -0.0000

Data Index: 1910041 (The Pile)

Max Activating Token Index: 268

Text #4

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0626. Min Act: -0.0000

Data Index: 714827 (The Pile)

Max Activating Token Index: 360

Text #5

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0623. Min Act: -0.0000

Data Index: 365467 (The Pile)

Max Activating Token Index: 651

Text #6

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0615. Min Act: -0.0000

Data Index: 905861 (The Pile)

Max Activating Token Index: 646

Text #7

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0614. Min Act: -0.0000

Data Index: 1636982 (The Pile)

Max Activating Token Index: 227

Text #8

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0614. Min Act: -0.0000

Data Index: 1414643 (The Pile)

Max Activating Token Index: 473

Text #9

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0606. Min Act: -0.0000

Data Index: 1404373 (The Pile)

Max Activating Token Index: 884

Text #10

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0605. Min Act: -0.0000

Data Index: 774584 (The Pile)

Max Activating Token Index: 39

Text #11

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0602. Min Act: -0.0000

Data Index: 852308 (The Pile)

Max Activating Token Index: 6

Text #12

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0600. Min Act: -0.0000

Data Index: 699955 (The Pile)

Max Activating Token Index: 18

Text #13

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0614. Min Act: -0.0000

Data Index: 1167839 (The Pile)

Max Activating Token Index: 264

Text #14

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0592. Min Act: -0.0000

Data Index: 698313 (The Pile)

Max Activating Token Index: 15

Text #15

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0590. Min Act: -0.0000

Data Index: 203894 (The Pile)

Max Activating Token Index: 58

Text #16

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0587. Min Act: -0.0000

Data Index: 1211756 (The Pile)

Max Activating Token Index: 40

Text #17

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0586. Min Act: -0.0000

Data Index: 1832667 (The Pile)

Max Activating Token Index: 401

Text #18

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0605. Min Act: -0.0000

Data Index: 419505 (The Pile)

Max Activating Token Index: 133

Text #19

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0584. Min Act: -0.0000

Data Index: 1235451 (The Pile)

Max Activating Token Index: 61