Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3515 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.5134. Min Range: -5.5134

Max Act: 5.5134. Min Act: -0.0000

Data Index: 429042 (The Pile)

Max Activating Token Index: 340

Text #1

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.9971. Min Act: -0.0000

Data Index: 1974806 (The Pile)

Max Activating Token Index: 385

Text #2

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.9364. Min Act: -0.0000

Data Index: 1651519 (The Pile)

Max Activating Token Index: 372

Text #3

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.8555. Min Act: -0.0000

Data Index: 1670169 (The Pile)

Max Activating Token Index: 331

Text #4

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.8571. Min Act: -0.0000

Data Index: 1667517 (The Pile)

Max Activating Token Index: 837

Text #5

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.4110. Min Act: -0.0000

Data Index: 1592891 (The Pile)

Max Activating Token Index: 643

Text #6

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.8841. Min Act: -0.0000

Data Index: 1369485 (The Pile)

Max Activating Token Index: 454

Text #7

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.2949. Min Act: -0.0000

Data Index: 733161 (The Pile)

Max Activating Token Index: 548

Text #8

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.3179. Min Act: -0.0000

Data Index: 161353 (The Pile)

Max Activating Token Index: 249

Text #9

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.4575. Min Act: -0.0000

Data Index: 1929588 (The Pile)

Max Activating Token Index: 571

Text #10

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.6422. Min Act: -0.0000

Data Index: 155309 (The Pile)

Max Activating Token Index: 502

Text #11

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.3722. Min Act: -0.0000

Data Index: 1193253 (The Pile)

Max Activating Token Index: 153

Text #12

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.3639. Min Act: -0.0000

Data Index: 644787 (The Pile)

Max Activating Token Index: 379

Text #13

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.3234. Min Act: -0.0000

Data Index: 1158003 (The Pile)

Max Activating Token Index: 509

Text #14

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.6189. Min Act: -0.0000

Data Index: 1754780 (The Pile)

Max Activating Token Index: 244

Text #15

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.7962. Min Act: -0.0000

Data Index: 1824864 (The Pile)

Max Activating Token Index: 185

Text #16

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.6640. Min Act: -0.0000

Data Index: 868939 (The Pile)

Max Activating Token Index: 223

Text #17

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.1710. Min Act: -0.0000

Data Index: 1468005 (The Pile)

Max Activating Token Index: 210

Text #18

Max Range: 5.5134. Min Range: -5.5134

Max Act: 4.1999. Min Act: -0.0000

Data Index: 466451 (The Pile)

Max Activating Token Index: 424

Text #19

Max Range: 5.5134. Min Range: -5.5134

Max Act: 3.9441. Min Act: -0.0000

Data Index: 952058 (The Pile)

Max Activating Token Index: 487