Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1854 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0937. Min Act: -0.0000

Data Index: 1620251 (The Pile)

Max Activating Token Index: 930

Text #1

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0674. Min Act: -0.0000

Data Index: 1343658 (The Pile)

Max Activating Token Index: 557

Text #2

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0646. Min Act: -0.0000

Data Index: 140255 (The Pile)

Max Activating Token Index: 417

Text #3

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0602. Min Act: -0.0000

Data Index: 1678434 (The Pile)

Max Activating Token Index: 647

Text #4

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0621. Min Act: -0.0000

Data Index: 509217 (The Pile)

Max Activating Token Index: 662

Text #5

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0544. Min Act: -0.0000

Data Index: 1672653 (The Pile)

Max Activating Token Index: 978

Text #6

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0595. Min Act: -0.0000

Data Index: 659613 (The Pile)

Max Activating Token Index: 941

Text #7

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0583. Min Act: -0.0000

Data Index: 611069 (The Pile)

Max Activating Token Index: 924

Text #8

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0505. Min Act: -0.0000

Data Index: 262115 (The Pile)

Max Activating Token Index: 770

Text #9

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0584. Min Act: -0.0000

Data Index: 1819673 (The Pile)

Max Activating Token Index: 236

Text #10

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0557. Min Act: -0.0000

Data Index: 913981 (The Pile)

Max Activating Token Index: 170

Text #11

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0556. Min Act: -0.0000

Data Index: 1428546 (The Pile)

Max Activating Token Index: 530

Text #12

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0535. Min Act: -0.0000

Data Index: 361763 (The Pile)

Max Activating Token Index: 959

Text #13

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0554. Min Act: -0.0000

Data Index: 242759 (The Pile)

Max Activating Token Index: 628

Text #14

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0510. Min Act: -0.0000

Data Index: 1907463 (The Pile)

Max Activating Token Index: 244

Text #15

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0600. Min Act: -0.0000

Data Index: 1061614 (The Pile)

Max Activating Token Index: 898

Text #16

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0486. Min Act: -0.0000

Data Index: 394489 (The Pile)

Max Activating Token Index: 471

Text #17

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0482. Min Act: -0.0000

Data Index: 1842722 (The Pile)

Max Activating Token Index: 510

Text #18

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0531. Min Act: -0.0000

Data Index: 718522 (The Pile)

Max Activating Token Index: 292

Text #19

Max Range: 0.0937. Min Range: -0.0937

Max Act: 0.0495. Min Act: -0.0000

Data Index: 1683599 (The Pile)

Max Activating Token Index: 132