Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 897 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.6520. Min Act: -0.0000

Data Index: 595775 (The Pile)

Max Activating Token Index: 138

Text #1

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.9910. Min Act: -0.0000

Data Index: 985251 (The Pile)

Max Activating Token Index: 354

Text #2

Max Range: 5.3665. Min Range: -5.3665

Max Act: 5.1560. Min Act: -0.0000

Data Index: 767350 (The Pile)

Max Activating Token Index: 484

Text #3

Max Range: 5.3665. Min Range: -5.3665

Max Act: 5.3665. Min Act: -0.0000

Data Index: 991940 (The Pile)

Max Activating Token Index: 176

Text #4

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.8598. Min Act: -0.0000

Data Index: 1014114 (The Pile)

Max Activating Token Index: 396

Text #5

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.9829. Min Act: -0.0000

Data Index: 1903276 (The Pile)

Max Activating Token Index: 491

Text #6

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.9955. Min Act: -0.0000

Data Index: 926689 (The Pile)

Max Activating Token Index: 329

Text #7

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.4943. Min Act: -0.0000

Data Index: 681367 (The Pile)

Max Activating Token Index: 520

Text #8

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.9784. Min Act: -0.0000

Data Index: 832976 (The Pile)

Max Activating Token Index: 308

Text #9

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.1970. Min Act: -0.0000

Data Index: 1822265 (The Pile)

Max Activating Token Index: 674

Text #10

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.8307. Min Act: -0.0000

Data Index: 1977245 (The Pile)

Max Activating Token Index: 429

Text #11

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.2980. Min Act: -0.0000

Data Index: 339391 (The Pile)

Max Activating Token Index: 580

Text #12

Max Range: 5.3665. Min Range: -5.3665

Max Act: 3.9672. Min Act: -0.0000

Data Index: 1542744 (The Pile)

Max Activating Token Index: 813

Text #13

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.2971. Min Act: -0.0000

Data Index: 1960281 (The Pile)

Max Activating Token Index: 571

Text #14

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.4205. Min Act: -0.0000

Data Index: 380862 (The Pile)

Max Activating Token Index: 634

Text #15

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.2732. Min Act: -0.0000

Data Index: 297844 (The Pile)

Max Activating Token Index: 713

Text #16

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.6127. Min Act: -0.0000

Data Index: 928964 (The Pile)

Max Activating Token Index: 590

Text #17

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.6887. Min Act: -0.0000

Data Index: 1558574 (The Pile)

Max Activating Token Index: 163

Text #18

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.3513. Min Act: -0.0000

Data Index: 385135 (The Pile)

Max Activating Token Index: 644

Text #19

Max Range: 5.3665. Min Range: -5.3665

Max Act: 4.3337. Min Act: -0.0000

Data Index: 891037 (The Pile)

Max Activating Token Index: 716