Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2120 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1701. Min Range: -1.1701

Max Act: 0.9262. Min Act: -0.0000

Data Index: 488283 (The Pile)

Max Activating Token Index: 588

Text #1

Max Range: 1.1701. Min Range: -1.1701

Max Act: 1.1106. Min Act: -0.0000

Data Index: 846564 (The Pile)

Max Activating Token Index: 771

Text #2

Max Range: 1.1701. Min Range: -1.1701

Max Act: 1.1364. Min Act: -0.0000

Data Index: 181334 (The Pile)

Max Activating Token Index: 418

Text #3

Max Range: 1.1701. Min Range: -1.1701

Max Act: 1.0886. Min Act: -0.0000

Data Index: 1916171 (The Pile)

Max Activating Token Index: 256

Text #4

Max Range: 1.1701. Min Range: -1.1701

Max Act: 1.1701. Min Act: -0.0000

Data Index: 649577 (The Pile)

Max Activating Token Index: 832

Text #5

Max Range: 1.1701. Min Range: -1.1701

Max Act: 1.0643. Min Act: -0.0000

Data Index: 1003798 (The Pile)

Max Activating Token Index: 966

Text #6

Max Range: 1.1701. Min Range: -1.1701

Max Act: 1.0757. Min Act: -0.0000

Data Index: 1825829 (The Pile)

Max Activating Token Index: 60

Text #7

Max Range: 1.1701. Min Range: -1.1701

Max Act: 1.0678. Min Act: -0.0000

Data Index: 1687792 (The Pile)

Max Activating Token Index: 770

Text #8

Max Range: 1.1701. Min Range: -1.1701

Max Act: 0.9966. Min Act: -0.0000

Data Index: 1618701 (The Pile)

Max Activating Token Index: 531

Text #9

Max Range: 1.1701. Min Range: -1.1701

Max Act: 1.0180. Min Act: -0.0000

Data Index: 906750 (The Pile)

Max Activating Token Index: 476

Text #10

Max Range: 1.1701. Min Range: -1.1701

Max Act: 1.0563. Min Act: -0.0000

Data Index: 1532118 (The Pile)

Max Activating Token Index: 901

Text #11

Max Range: 1.1701. Min Range: -1.1701

Max Act: 0.9839. Min Act: -0.0000

Data Index: 193820 (The Pile)

Max Activating Token Index: 76

Text #12

Max Range: 1.1701. Min Range: -1.1701

Max Act: 0.9908. Min Act: -0.0000

Data Index: 1099978 (The Pile)

Max Activating Token Index: 256

Text #13

Max Range: 1.1701. Min Range: -1.1701

Max Act: 0.9493. Min Act: -0.0000

Data Index: 1254231 (The Pile)

Max Activating Token Index: 661

Text #14

Max Range: 1.1701. Min Range: -1.1701

Max Act: 1.0322. Min Act: -0.0000

Data Index: 1673082 (The Pile)

Max Activating Token Index: 215

Text #15

Max Range: 1.1701. Min Range: -1.1701

Max Act: 0.9381. Min Act: -0.0000

Data Index: 1283605 (The Pile)

Max Activating Token Index: 104

Text #16

Max Range: 1.1701. Min Range: -1.1701

Max Act: 1.1143. Min Act: -0.0000

Data Index: 1636633 (The Pile)

Max Activating Token Index: 914

Text #17

Max Range: 1.1701. Min Range: -1.1701

Max Act: 0.9617. Min Act: -0.0000

Data Index: 1069572 (The Pile)

Max Activating Token Index: 46

Text #18

Max Range: 1.1701. Min Range: -1.1701

Max Act: 1.1199. Min Act: -0.0000

Data Index: 1734579 (The Pile)

Max Activating Token Index: 983

Text #19

Max Range: 1.1701. Min Range: -1.1701

Max Act: 0.8835. Min Act: -0.0000

Data Index: 750275 (The Pile)

Max Activating Token Index: 117