Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2417 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.8911. Min Act: -0.0000

Data Index: 1923811 (The Pile)

Max Activating Token Index: 72

Text #1

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.6366. Min Act: -0.0000

Data Index: 145577 (The Pile)

Max Activating Token Index: 735

Text #2

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.6294. Min Act: -0.0000

Data Index: 91123 (The Pile)

Max Activating Token Index: 177

Text #3

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.6393. Min Act: -0.0000

Data Index: 864784 (The Pile)

Max Activating Token Index: 412

Text #4

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.6247. Min Act: -0.0000

Data Index: 906605 (The Pile)

Max Activating Token Index: 470

Text #5

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.6213. Min Act: -0.0000

Data Index: 57032 (The Pile)

Max Activating Token Index: 291

Text #6

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.6071. Min Act: -0.0000

Data Index: 249023 (The Pile)

Max Activating Token Index: 496

Text #7

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.6003. Min Act: -0.0000

Data Index: 1904524 (The Pile)

Max Activating Token Index: 583

Text #8

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.5889. Min Act: -0.0000

Data Index: 1291524 (The Pile)

Max Activating Token Index: 320

Text #9

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.6171. Min Act: -0.0000

Data Index: 1020223 (The Pile)

Max Activating Token Index: 842

Text #10

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.5752. Min Act: -0.0000

Data Index: 771844 (The Pile)

Max Activating Token Index: 546

Text #11

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.5771. Min Act: -0.0000

Data Index: 709518 (The Pile)

Max Activating Token Index: 808

Text #12

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.5828. Min Act: -0.0000

Data Index: 875509 (The Pile)

Max Activating Token Index: 637

Text #13

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.5845. Min Act: -0.0000

Data Index: 626344 (The Pile)

Max Activating Token Index: 397

Text #14

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.5766. Min Act: -0.0000

Data Index: 1206712 (The Pile)

Max Activating Token Index: 142

Text #15

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.5792. Min Act: -0.0000

Data Index: 1534258 (The Pile)

Max Activating Token Index: 674

Text #16

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.6072. Min Act: -0.0000

Data Index: 1557723 (The Pile)

Max Activating Token Index: 885

Text #17

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.6103. Min Act: -0.0000

Data Index: 1648612 (The Pile)

Max Activating Token Index: 80

Text #18

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.5684. Min Act: -0.0000

Data Index: 1956927 (The Pile)

Max Activating Token Index: 437

Text #19

Max Range: 0.8911. Min Range: -0.8911

Max Act: 0.5481. Min Act: -0.0000

Data Index: 1682275 (The Pile)

Max Activating Token Index: 352