Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3127 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.9014. Min Act: -0.0000

Data Index: 1968101 (The Pile)

Max Activating Token Index: 371

Text #1

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.7570. Min Act: -0.0000

Data Index: 1451153 (The Pile)

Max Activating Token Index: 543

Text #2

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.5780. Min Act: -0.0000

Data Index: 523696 (The Pile)

Max Activating Token Index: 976

Text #3

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.6356. Min Act: -0.0000

Data Index: 1455237 (The Pile)

Max Activating Token Index: 206

Text #4

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.6213. Min Act: -0.0000

Data Index: 60688 (The Pile)

Max Activating Token Index: 436

Text #5

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.5084. Min Act: -0.0000

Data Index: 198133 (The Pile)

Max Activating Token Index: 323

Text #6

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.6499. Min Act: -0.0000

Data Index: 1383613 (The Pile)

Max Activating Token Index: 961

Text #7

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.6451. Min Act: -0.0000

Data Index: 1715484 (The Pile)

Max Activating Token Index: 140

Text #8

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.5222. Min Act: -0.0000

Data Index: 1328392 (The Pile)

Max Activating Token Index: 213

Text #9

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.5227. Min Act: -0.0000

Data Index: 1292988 (The Pile)

Max Activating Token Index: 897

Text #10

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.5153. Min Act: -0.0000

Data Index: 956628 (The Pile)

Max Activating Token Index: 721

Text #11

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.5056. Min Act: -0.0000

Data Index: 600890 (The Pile)

Max Activating Token Index: 921

Text #12

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.4878. Min Act: -0.0000

Data Index: 12282 (The Pile)

Max Activating Token Index: 421

Text #13

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.4887. Min Act: -0.0000

Data Index: 298217 (The Pile)

Max Activating Token Index: 200

Text #14

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.4807. Min Act: -0.0000

Data Index: 1488918 (The Pile)

Max Activating Token Index: 866

Text #15

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.4776. Min Act: -0.0000

Data Index: 729516 (The Pile)

Max Activating Token Index: 583

Text #16

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.4768. Min Act: -0.0000

Data Index: 1545739 (The Pile)

Max Activating Token Index: 682

Text #17

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.4828. Min Act: -0.0000

Data Index: 818174 (The Pile)

Max Activating Token Index: 545

Text #18

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.3835. Min Act: -0.0000

Data Index: 242533 (The Pile)

Max Activating Token Index: 763

Text #19

Max Range: 1.9014. Min Range: -1.9014

Max Act: 1.4456. Min Act: -0.0000

Data Index: 1940569 (The Pile)

Max Activating Token Index: 436