Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3139 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.6582. Min Act: -0.0000

Data Index: 1765479 (The Pile)

Max Activating Token Index: 620

Text #1

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.5103. Min Act: -0.0000

Data Index: 1558876 (The Pile)

Max Activating Token Index: 484

Text #2

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.3004. Min Act: -0.0000

Data Index: 1914866 (The Pile)

Max Activating Token Index: 293

Text #3

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.3613. Min Act: -0.0000

Data Index: 205147 (The Pile)

Max Activating Token Index: 968

Text #4

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.4393. Min Act: -0.0000

Data Index: 935073 (The Pile)

Max Activating Token Index: 648

Text #5

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.3498. Min Act: -0.0000

Data Index: 429156 (The Pile)

Max Activating Token Index: 94

Text #6

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.1959. Min Act: -0.0000

Data Index: 634633 (The Pile)

Max Activating Token Index: 10

Text #7

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.2500. Min Act: -0.0000

Data Index: 790423 (The Pile)

Max Activating Token Index: 473

Text #8

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.3615. Min Act: -0.0000

Data Index: 529975 (The Pile)

Max Activating Token Index: 386

Text #9

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.2124. Min Act: -0.0000

Data Index: 155109 (The Pile)

Max Activating Token Index: 700

Text #10

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.1499. Min Act: -0.0000

Data Index: 986270 (The Pile)

Max Activating Token Index: 848

Text #11

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.2464. Min Act: -0.0000

Data Index: 1400162 (The Pile)

Max Activating Token Index: 847

Text #12

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.0948. Min Act: -0.0000

Data Index: 330631 (The Pile)

Max Activating Token Index: 925

Text #13

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.1867. Min Act: -0.0000

Data Index: 1921205 (The Pile)

Max Activating Token Index: 670

Text #14

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.1769. Min Act: -0.0000

Data Index: 165456 (The Pile)

Max Activating Token Index: 992

Text #15

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.2000. Min Act: -0.0000

Data Index: 556407 (The Pile)

Max Activating Token Index: 247

Text #16

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.1706. Min Act: -0.0000

Data Index: 1869462 (The Pile)

Max Activating Token Index: 989

Text #17

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.2163. Min Act: -0.0000

Data Index: 1588491 (The Pile)

Max Activating Token Index: 337

Text #18

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.1616. Min Act: -0.0000

Data Index: 903634 (The Pile)

Max Activating Token Index: 922

Text #19

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.1769. Min Act: -0.0000

Data Index: 1861760 (The Pile)

Max Activating Token Index: 979