Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4351 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.6542. Min Act: -0.0000

Data Index: 1564608 (The Pile)

Max Activating Token Index: 396

Text #1

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.7656. Min Act: -0.0000

Data Index: 1270523 (The Pile)

Max Activating Token Index: 504

Text #2

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.5204. Min Act: -0.0000

Data Index: 1354783 (The Pile)

Max Activating Token Index: 756

Text #3

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.6306. Min Act: -0.0000

Data Index: 811122 (The Pile)

Max Activating Token Index: 714

Text #4

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.4932. Min Act: -0.0000

Data Index: 1003937 (The Pile)

Max Activating Token Index: 173

Text #5

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.4305. Min Act: -0.0000

Data Index: 1040744 (The Pile)

Max Activating Token Index: 973

Text #6

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.4381. Min Act: -0.0000

Data Index: 577687 (The Pile)

Max Activating Token Index: 788

Text #7

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.4214. Min Act: -0.0000

Data Index: 946606 (The Pile)

Max Activating Token Index: 833

Text #8

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.3976. Min Act: -0.0000

Data Index: 151020 (The Pile)

Max Activating Token Index: 476

Text #9

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.3815. Min Act: -0.0000

Data Index: 272898 (The Pile)

Max Activating Token Index: 377

Text #10

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.3815. Min Act: -0.0000

Data Index: 1375 (The Pile)

Max Activating Token Index: 767

Text #11

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.3831. Min Act: -0.0000

Data Index: 203540 (The Pile)

Max Activating Token Index: 963

Text #12

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.3792. Min Act: -0.0000

Data Index: 1296400 (The Pile)

Max Activating Token Index: 62

Text #13

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.3606. Min Act: -0.0000

Data Index: 1987652 (The Pile)

Max Activating Token Index: 511

Text #14

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.3700. Min Act: -0.0000

Data Index: 233660 (The Pile)

Max Activating Token Index: 439

Text #15

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.3613. Min Act: -0.0000

Data Index: 1844238 (The Pile)

Max Activating Token Index: 406

Text #16

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.3589. Min Act: -0.0000

Data Index: 598037 (The Pile)

Max Activating Token Index: 327

Text #17

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.3538. Min Act: -0.0000

Data Index: 863144 (The Pile)

Max Activating Token Index: 241

Text #18

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.3625. Min Act: -0.0000

Data Index: 150865 (The Pile)

Max Activating Token Index: 415

Text #19

Max Range: 0.7656. Min Range: -0.7656

Max Act: 0.3340. Min Act: 0.0000

Data Index: 1375917 (The Pile)

Max Activating Token Index: 882