Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2915 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5655. Min Range: -1.5655

Max Act: 1.5655. Min Act: -0.0000

Data Index: 178303 (The Pile)

Max Activating Token Index: 877

Text #1

Max Range: 1.5655. Min Range: -1.5655

Max Act: 1.3427. Min Act: -0.0000

Data Index: 504028 (The Pile)

Max Activating Token Index: 807

Text #2

Max Range: 1.5655. Min Range: -1.5655

Max Act: 1.2094. Min Act: -0.0000

Data Index: 688786 (The Pile)

Max Activating Token Index: 191

Text #3

Max Range: 1.5655. Min Range: -1.5655

Max Act: 1.2584. Min Act: -0.0000

Data Index: 399134 (The Pile)

Max Activating Token Index: 448

Text #4

Max Range: 1.5655. Min Range: -1.5655

Max Act: 1.2922. Min Act: -0.0000

Data Index: 555100 (The Pile)

Max Activating Token Index: 591

Text #5

Max Range: 1.5655. Min Range: -1.5655

Max Act: 1.1411. Min Act: 0.0000

Data Index: 548097 (The Pile)

Max Activating Token Index: 325

Text #6

Max Range: 1.5655. Min Range: -1.5655

Max Act: 1.0558. Min Act: -0.0000

Data Index: 467390 (The Pile)

Max Activating Token Index: 988

Text #7

Max Range: 1.5655. Min Range: -1.5655

Max Act: 1.0497. Min Act: 0.0000

Data Index: 201005 (The Pile)

Max Activating Token Index: 803

Text #8

Max Range: 1.5655. Min Range: -1.5655

Max Act: 0.9703. Min Act: 0.0000

Data Index: 1006636 (The Pile)

Max Activating Token Index: 634

Text #9

Max Range: 1.5655. Min Range: -1.5655

Max Act: 1.0355. Min Act: -0.0000

Data Index: 177815 (The Pile)

Max Activating Token Index: 337

Text #10

Max Range: 1.5655. Min Range: -1.5655

Max Act: 0.9678. Min Act: 0.0000

Data Index: 704265 (The Pile)

Max Activating Token Index: 634

Text #11

Max Range: 1.5655. Min Range: -1.5655

Max Act: 0.9899. Min Act: 0.0000

Data Index: 1133134 (The Pile)

Max Activating Token Index: 323

Text #12

Max Range: 1.5655. Min Range: -1.5655

Max Act: 0.9823. Min Act: -0.0000

Data Index: 1668126 (The Pile)

Max Activating Token Index: 417

Text #13

Max Range: 1.5655. Min Range: -1.5655

Max Act: 0.9768. Min Act: -0.0000

Data Index: 1081890 (The Pile)

Max Activating Token Index: 778

Text #14

Max Range: 1.5655. Min Range: -1.5655

Max Act: 0.9676. Min Act: -0.0000

Data Index: 1098955 (The Pile)

Max Activating Token Index: 632

Text #15

Max Range: 1.5655. Min Range: -1.5655

Max Act: 0.9614. Min Act: -0.0000

Data Index: 1989711 (The Pile)

Max Activating Token Index: 268

Text #16

Max Range: 1.5655. Min Range: -1.5655

Max Act: 1.0055. Min Act: -0.0000

Data Index: 1223957 (The Pile)

Max Activating Token Index: 833

Text #17

Max Range: 1.5655. Min Range: -1.5655

Max Act: 0.9300. Min Act: -0.0000

Data Index: 1042658 (The Pile)

Max Activating Token Index: 148

Text #18

Max Range: 1.5655. Min Range: -1.5655

Max Act: 0.9254. Min Act: -0.0000

Data Index: 1423503 (The Pile)

Max Activating Token Index: 72

Text #19

Max Range: 1.5655. Min Range: -1.5655

Max Act: 0.8656. Min Act: -0.0000

Data Index: 1388435 (The Pile)

Max Activating Token Index: 379