Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2354 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.8796. Min Act: -0.0000

Data Index: 429466 (The Pile)

Max Activating Token Index: 780

Text #1

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.7703. Min Act: -0.0000

Data Index: 1150384 (The Pile)

Max Activating Token Index: 383

Text #2

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.6088. Min Act: -0.0000

Data Index: 449736 (The Pile)

Max Activating Token Index: 861

Text #3

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.8861. Min Act: -0.0000

Data Index: 907869 (The Pile)

Max Activating Token Index: 962

Text #4

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.7223. Min Act: -0.0000

Data Index: 1632251 (The Pile)

Max Activating Token Index: 1017

Text #5

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.7356. Min Act: -0.0000

Data Index: 686250 (The Pile)

Max Activating Token Index: 850

Text #6

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.7158. Min Act: -0.0000

Data Index: 1798075 (The Pile)

Max Activating Token Index: 582

Text #7

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.8439. Min Act: -0.0000

Data Index: 1755325 (The Pile)

Max Activating Token Index: 1014

Text #8

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.6077. Min Act: -0.0000

Data Index: 693908 (The Pile)

Max Activating Token Index: 363

Text #9

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.8682. Min Act: -0.0000

Data Index: 1356869 (The Pile)

Max Activating Token Index: 916

Text #10

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.3655. Min Act: -0.0000

Data Index: 164775 (The Pile)

Max Activating Token Index: 762

Text #11

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.5754. Min Act: -0.0000

Data Index: 1053736 (The Pile)

Max Activating Token Index: 461

Text #12

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.6635. Min Act: -0.0000

Data Index: 286596 (The Pile)

Max Activating Token Index: 45

Text #13

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.5426. Min Act: -0.0000

Data Index: 384507 (The Pile)

Max Activating Token Index: 662

Text #14

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.6620. Min Act: -0.0000

Data Index: 1057497 (The Pile)

Max Activating Token Index: 458

Text #15

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.3715. Min Act: -0.0000

Data Index: 1081148 (The Pile)

Max Activating Token Index: 917

Text #16

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.6052. Min Act: -0.0000

Data Index: 1384500 (The Pile)

Max Activating Token Index: 502

Text #17

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.5109. Min Act: -0.0000

Data Index: 224138 (The Pile)

Max Activating Token Index: 355

Text #18

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.4483. Min Act: -0.0000

Data Index: 1509123 (The Pile)

Max Activating Token Index: 750

Text #19

Max Range: 1.8861. Min Range: -1.8861

Max Act: 1.3617. Min Act: -0.0000

Data Index: 478915 (The Pile)

Max Activating Token Index: 866