Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2461 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.6607. Min Act: -0.0000

Data Index: 1510348 (The Pile)

Max Activating Token Index: 410

Text #1

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.6006. Min Act: -0.0000

Data Index: 1156942 (The Pile)

Max Activating Token Index: 338

Text #2

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.5536. Min Act: -0.0000

Data Index: 744836 (The Pile)

Max Activating Token Index: 109

Text #3

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.6250. Min Act: -0.0000

Data Index: 1005021 (The Pile)

Max Activating Token Index: 593

Text #4

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.4703. Min Act: -0.0000

Data Index: 649449 (The Pile)

Max Activating Token Index: 430

Text #5

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.4471. Min Act: -0.0000

Data Index: 532834 (The Pile)

Max Activating Token Index: 335

Text #6

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.4289. Min Act: -0.0000

Data Index: 1444565 (The Pile)

Max Activating Token Index: 602

Text #7

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.3792. Min Act: -0.0000

Data Index: 1058783 (The Pile)

Max Activating Token Index: 411

Text #8

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.2852. Min Act: -0.0000

Data Index: 18492 (The Pile)

Max Activating Token Index: 197

Text #9

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.2139. Min Act: -0.0000

Data Index: 323805 (The Pile)

Max Activating Token Index: 925

Text #10

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.2668. Min Act: -0.0000

Data Index: 314249 (The Pile)

Max Activating Token Index: 449

Text #11

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.2110. Min Act: -0.0000

Data Index: 407935 (The Pile)

Max Activating Token Index: 113

Text #12

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.0703. Min Act: -0.0000

Data Index: 1946722 (The Pile)

Max Activating Token Index: 57

Text #13

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.3514. Min Act: -0.0000

Data Index: 1230810 (The Pile)

Max Activating Token Index: 907

Text #14

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.2993. Min Act: -0.0000

Data Index: 1384278 (The Pile)

Max Activating Token Index: 218

Text #15

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.2458. Min Act: -0.0000

Data Index: 480620 (The Pile)

Max Activating Token Index: 731

Text #16

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.2283. Min Act: -0.0000

Data Index: 956091 (The Pile)

Max Activating Token Index: 350

Text #17

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.1739. Min Act: -0.0000

Data Index: 10353 (The Pile)

Max Activating Token Index: 113

Text #18

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.1709. Min Act: -0.0000

Data Index: 1345864 (The Pile)

Max Activating Token Index: 750

Text #19

Max Range: 1.6607. Min Range: -1.6607

Max Act: 1.3036. Min Act: -0.0000

Data Index: 1992292 (The Pile)

Max Activating Token Index: 189