Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2175 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.3349. Min Act: -0.0000

Data Index: 1489622 (The Pile)

Max Activating Token Index: 756

Text #1

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.3121. Min Act: -0.0000

Data Index: 1859459 (The Pile)

Max Activating Token Index: 572

Text #2

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.3732. Min Act: -0.0000

Data Index: 300987 (The Pile)

Max Activating Token Index: 291

Text #3

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.4955. Min Act: -0.0000

Data Index: 449730 (The Pile)

Max Activating Token Index: 127

Text #4

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.4001. Min Act: -0.0000

Data Index: 1202077 (The Pile)

Max Activating Token Index: 981

Text #5

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.3803. Min Act: -0.0000

Data Index: 639404 (The Pile)

Max Activating Token Index: 646

Text #6

Max Range: 1.4955. Min Range: -1.4955

Max Act: 0.9653. Min Act: -0.0000

Data Index: 1976401 (The Pile)

Max Activating Token Index: 960

Text #7

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.2461. Min Act: -0.0000

Data Index: 986343 (The Pile)

Max Activating Token Index: 654

Text #8

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.1189. Min Act: -0.0000

Data Index: 1778596 (The Pile)

Max Activating Token Index: 884

Text #9

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.1539. Min Act: -0.0000

Data Index: 1869134 (The Pile)

Max Activating Token Index: 904

Text #10

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.1029. Min Act: -0.0000

Data Index: 1058864 (The Pile)

Max Activating Token Index: 660

Text #11

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.0661. Min Act: -0.0000

Data Index: 74568 (The Pile)

Max Activating Token Index: 845

Text #12

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.0854. Min Act: -0.0000

Data Index: 1368042 (The Pile)

Max Activating Token Index: 465

Text #13

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.0697. Min Act: -0.0000

Data Index: 1151058 (The Pile)

Max Activating Token Index: 810

Text #14

Max Range: 1.4955. Min Range: -1.4955

Max Act: 0.9361. Min Act: -0.0000

Data Index: 785527 (The Pile)

Max Activating Token Index: 660

Text #15

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.0945. Min Act: -0.0000

Data Index: 529808 (The Pile)

Max Activating Token Index: 92

Text #16

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.2095. Min Act: -0.0000

Data Index: 1842874 (The Pile)

Max Activating Token Index: 902

Text #17

Max Range: 1.4955. Min Range: -1.4955

Max Act: 0.8941. Min Act: -0.0000

Data Index: 1881886 (The Pile)

Max Activating Token Index: 159

Text #18

Max Range: 1.4955. Min Range: -1.4955

Max Act: 0.9665. Min Act: -0.0000

Data Index: 1267426 (The Pile)

Max Activating Token Index: 383

Text #19

Max Range: 1.4955. Min Range: -1.4955

Max Act: 1.2351. Min Act: -0.0000

Data Index: 664792 (The Pile)

Max Activating Token Index: 838