Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3909 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0689. Min Act: 0.0000

Data Index: 856738 (The Pile)

Max Activating Token Index: 1023

Text #1

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0656. Min Act: 0.0000

Data Index: 445821 (The Pile)

Max Activating Token Index: 1023

Text #2

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0647. Min Act: 0.0000

Data Index: 847225 (The Pile)

Max Activating Token Index: 1023

Text #3

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0632. Min Act: -0.0000

Data Index: 168173 (The Pile)

Max Activating Token Index: 1023

Text #4

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0630. Min Act: 0.0000

Data Index: 465793 (The Pile)

Max Activating Token Index: 1023

Text #5

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0630. Min Act: 0.0000

Data Index: 190093 (The Pile)

Max Activating Token Index: 1023

Text #6

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0624. Min Act: 0.0000

Data Index: 49205 (The Pile)

Max Activating Token Index: 1023

Text #7

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0622. Min Act: 0.0000

Data Index: 1287981 (The Pile)

Max Activating Token Index: 1023

Text #8

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0618. Min Act: 0.0000

Data Index: 1255695 (The Pile)

Max Activating Token Index: 1023

Text #9

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0599. Min Act: 0.0000

Data Index: 438375 (The Pile)

Max Activating Token Index: 1023

Text #10

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0594. Min Act: 0.0000

Data Index: 1609832 (The Pile)

Max Activating Token Index: 1023

Text #11

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0594. Min Act: 0.0000

Data Index: 1727117 (The Pile)

Max Activating Token Index: 1023

Text #12

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0575. Min Act: -0.0000

Data Index: 1454061 (The Pile)

Max Activating Token Index: 1023

Text #13

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0573. Min Act: 0.0000

Data Index: 1922576 (The Pile)

Max Activating Token Index: 914

Text #14

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0568. Min Act: 0.0000

Data Index: 894517 (The Pile)

Max Activating Token Index: 1023

Text #15

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0568. Min Act: -0.0000

Data Index: 1896976 (The Pile)

Max Activating Token Index: 1023

Text #16

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0544. Min Act: 0.0000

Data Index: 798133 (The Pile)

Max Activating Token Index: 1023

Text #17

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0562. Min Act: -0.0000

Data Index: 1037755 (The Pile)

Max Activating Token Index: 1023

Text #18

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0562. Min Act: 0.0000

Data Index: 1770985 (The Pile)

Max Activating Token Index: 537

Text #19

Max Range: 0.0689. Min Range: -0.0689

Max Act: 0.0559. Min Act: 0.0000

Data Index: 1036516 (The Pile)

Max Activating Token Index: 1023