Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4443 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.3910. Min Range: -1.3910

Max Act: 1.3910. Min Act: -0.0000

Data Index: 1123970 (The Pile)

Max Activating Token Index: 328

Text #1

Max Range: 1.3910. Min Range: -1.3910

Max Act: 1.0857. Min Act: -0.0000

Data Index: 917954 (The Pile)

Max Activating Token Index: 157

Text #2

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.9736. Min Act: -0.0000

Data Index: 375667 (The Pile)

Max Activating Token Index: 914

Text #3

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.9751. Min Act: -0.0000

Data Index: 744865 (The Pile)

Max Activating Token Index: 238

Text #4

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.9491. Min Act: -0.0000

Data Index: 1578582 (The Pile)

Max Activating Token Index: 1007

Text #5

Max Range: 1.3910. Min Range: -1.3910

Max Act: 1.0581. Min Act: -0.0000

Data Index: 364409 (The Pile)

Max Activating Token Index: 273

Text #6

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.9555. Min Act: -0.0000

Data Index: 279352 (The Pile)

Max Activating Token Index: 426

Text #7

Max Range: 1.3910. Min Range: -1.3910

Max Act: 1.0239. Min Act: -0.0000

Data Index: 565591 (The Pile)

Max Activating Token Index: 863

Text #8

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.9010. Min Act: -0.0000

Data Index: 234730 (The Pile)

Max Activating Token Index: 650

Text #9

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.9364. Min Act: -0.0000

Data Index: 296510 (The Pile)

Max Activating Token Index: 901

Text #10

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.8918. Min Act: -0.0000

Data Index: 1047934 (The Pile)

Max Activating Token Index: 279

Text #11

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.9695. Min Act: -0.0000

Data Index: 1286043 (The Pile)

Max Activating Token Index: 641

Text #12

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.9197. Min Act: -0.0000

Data Index: 1606587 (The Pile)

Max Activating Token Index: 259

Text #13

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.8916. Min Act: -0.0000

Data Index: 1179 (The Pile)

Max Activating Token Index: 471

Text #14

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.9105. Min Act: -0.0000

Data Index: 1604688 (The Pile)

Max Activating Token Index: 433

Text #15

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.7768. Min Act: -0.0000

Data Index: 519998 (The Pile)

Max Activating Token Index: 847

Text #16

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.9082. Min Act: -0.0000

Data Index: 1578106 (The Pile)

Max Activating Token Index: 616

Text #17

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.8541. Min Act: -0.0000

Data Index: 1269094 (The Pile)

Max Activating Token Index: 1022

Text #18

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.8951. Min Act: -0.0000

Data Index: 1714901 (The Pile)

Max Activating Token Index: 120

Text #19

Max Range: 1.3910. Min Range: -1.3910

Max Act: 0.8770. Min Act: -0.0000

Data Index: 902887 (The Pile)

Max Activating Token Index: 536