Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 4078 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.0387. Min Range: -1.0387

Max Act: 1.0387. Min Act: -0.0000

Data Index: 105945 (The Pile)

Max Activating Token Index: 520

Text #1

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.9887. Min Act: -0.0000

Data Index: 1132685 (The Pile)

Max Activating Token Index: 479

Text #2

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.8046. Min Act: -0.0000

Data Index: 1149240 (The Pile)

Max Activating Token Index: 486

Text #3

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.7769. Min Act: -0.0000

Data Index: 1172087 (The Pile)

Max Activating Token Index: 309

Text #4

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.8340. Min Act: -0.0000

Data Index: 266406 (The Pile)

Max Activating Token Index: 758

Text #5

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.7425. Min Act: -0.0000

Data Index: 1488390 (The Pile)

Max Activating Token Index: 891

Text #6

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.7402. Min Act: -0.0000

Data Index: 1681489 (The Pile)

Max Activating Token Index: 730

Text #7

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.7028. Min Act: -0.0000

Data Index: 619278 (The Pile)

Max Activating Token Index: 450

Text #8

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.7381. Min Act: -0.0000

Data Index: 74597 (The Pile)

Max Activating Token Index: 488

Text #9

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.7370. Min Act: -0.0000

Data Index: 1655797 (The Pile)

Max Activating Token Index: 520

Text #10

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.7311. Min Act: -0.0000

Data Index: 1478193 (The Pile)

Max Activating Token Index: 661

Text #11

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.7318. Min Act: -0.0000

Data Index: 1411733 (The Pile)

Max Activating Token Index: 356

Text #12

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.6807. Min Act: -0.0000

Data Index: 229756 (The Pile)

Max Activating Token Index: 417

Text #13

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.7115. Min Act: -0.0000

Data Index: 1748747 (The Pile)

Max Activating Token Index: 115

Text #14

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.7111. Min Act: -0.0000

Data Index: 449657 (The Pile)

Max Activating Token Index: 256

Text #15

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.6810. Min Act: -0.0000

Data Index: 71206 (The Pile)

Max Activating Token Index: 305

Text #16

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.6793. Min Act: -0.0000

Data Index: 357372 (The Pile)

Max Activating Token Index: 120

Text #17

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.7185. Min Act: -0.0000

Data Index: 108951 (The Pile)

Max Activating Token Index: 644

Text #18

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.6717. Min Act: -0.0000

Data Index: 1231435 (The Pile)

Max Activating Token Index: 382

Text #19

Max Range: 1.0387. Min Range: -1.0387

Max Act: 0.6759. Min Act: -0.0000

Data Index: 504162 (The Pile)

Max Activating Token Index: 453