Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2635 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 4.0615. Min Range: -4.0615

Max Act: 4.0615. Min Act: -0.0000

Data Index: 1125223 (The Pile)

Max Activating Token Index: 326

Text #1

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.7582. Min Act: -0.0000

Data Index: 907052 (The Pile)

Max Activating Token Index: 88

Text #2

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.8492. Min Act: -0.0000

Data Index: 502492 (The Pile)

Max Activating Token Index: 567

Text #3

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.5478. Min Act: -0.0000

Data Index: 1346431 (The Pile)

Max Activating Token Index: 135

Text #4

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.3954. Min Act: -0.0000

Data Index: 660587 (The Pile)

Max Activating Token Index: 530

Text #5

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.5656. Min Act: -0.0000

Data Index: 1229286 (The Pile)

Max Activating Token Index: 176

Text #6

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.4225. Min Act: -0.0000

Data Index: 1286574 (The Pile)

Max Activating Token Index: 24

Text #7

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.8318. Min Act: -0.0000

Data Index: 1799875 (The Pile)

Max Activating Token Index: 125

Text #8

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.2015. Min Act: -0.0000

Data Index: 617966 (The Pile)

Max Activating Token Index: 56

Text #9

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.3459. Min Act: -0.0000

Data Index: 549953 (The Pile)

Max Activating Token Index: 540

Text #10

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.3872. Min Act: -0.0000

Data Index: 179240 (The Pile)

Max Activating Token Index: 889

Text #11

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.3153. Min Act: -0.0000

Data Index: 254839 (The Pile)

Max Activating Token Index: 80

Text #12

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.0468. Min Act: -0.0000

Data Index: 1530035 (The Pile)

Max Activating Token Index: 331

Text #13

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.4153. Min Act: -0.0000

Data Index: 1234226 (The Pile)

Max Activating Token Index: 306

Text #14

Max Range: 4.0615. Min Range: -4.0615

Max Act: 2.8560. Min Act: -0.0000

Data Index: 1079001 (The Pile)

Max Activating Token Index: 146

Text #15

Max Range: 4.0615. Min Range: -4.0615

Max Act: 2.9704. Min Act: -0.0000

Data Index: 1875629 (The Pile)

Max Activating Token Index: 130

Text #16

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.2513. Min Act: -0.0000

Data Index: 1165686 (The Pile)

Max Activating Token Index: 82

Text #17

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.1750. Min Act: -0.0000

Data Index: 855255 (The Pile)

Max Activating Token Index: 141

Text #18

Max Range: 4.0615. Min Range: -4.0615

Max Act: 3.1537. Min Act: -0.0000

Data Index: 1982377 (The Pile)

Max Activating Token Index: 337

Text #19

Max Range: 4.0615. Min Range: -4.0615

Max Act: 2.3100. Min Act: -0.0000

Data Index: 1858862 (The Pile)

Max Activating Token Index: 857