Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2042 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2917. Min Act: -0.0000

Data Index: 1136445 (The Pile)

Max Activating Token Index: 75

Text #1

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2999. Min Act: -0.0000

Data Index: 1497992 (The Pile)

Max Activating Token Index: 261

Text #2

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2963. Min Act: -0.0000

Data Index: 262867 (The Pile)

Max Activating Token Index: 661

Text #3

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.3076. Min Act: -0.0000

Data Index: 452633 (The Pile)

Max Activating Token Index: 737

Text #4

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2330. Min Act: -0.0000

Data Index: 438799 (The Pile)

Max Activating Token Index: 656

Text #5

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2751. Min Act: -0.0000

Data Index: 1609984 (The Pile)

Max Activating Token Index: 176

Text #6

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2667. Min Act: -0.0000

Data Index: 1599885 (The Pile)

Max Activating Token Index: 450

Text #7

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2895. Min Act: -0.0000

Data Index: 110907 (The Pile)

Max Activating Token Index: 918

Text #8

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2522. Min Act: -0.0000

Data Index: 984127 (The Pile)

Max Activating Token Index: 252

Text #9

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2575. Min Act: -0.0000

Data Index: 183905 (The Pile)

Max Activating Token Index: 140

Text #10

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2732. Min Act: -0.0000

Data Index: 191767 (The Pile)

Max Activating Token Index: 938

Text #11

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2548. Min Act: -0.0000

Data Index: 1966411 (The Pile)

Max Activating Token Index: 322

Text #12

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2444. Min Act: -0.0000

Data Index: 163415 (The Pile)

Max Activating Token Index: 685

Text #13

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2516. Min Act: -0.0000

Data Index: 1912259 (The Pile)

Max Activating Token Index: 793

Text #14

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2619. Min Act: -0.0000

Data Index: 923230 (The Pile)

Max Activating Token Index: 593

Text #15

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2273. Min Act: -0.0000

Data Index: 33427 (The Pile)

Max Activating Token Index: 571

Text #16

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2753. Min Act: -0.0000

Data Index: 859481 (The Pile)

Max Activating Token Index: 390

Text #17

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2493. Min Act: -0.0000

Data Index: 1655171 (The Pile)

Max Activating Token Index: 396

Text #18

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2455. Min Act: -0.0000

Data Index: 1678039 (The Pile)

Max Activating Token Index: 429

Text #19

Max Range: 0.3076. Min Range: -0.3076

Max Act: 0.2437. Min Act: -0.0000

Data Index: 405797 (The Pile)

Max Activating Token Index: 39