Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4390 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.7181. Min Act: -0.0000

Data Index: 922833 (The Pile)

Max Activating Token Index: 821

Text #1

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.4058. Min Act: -0.0000

Data Index: 1591470 (The Pile)

Max Activating Token Index: 327

Text #2

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.7043. Min Act: -0.0000

Data Index: 1933162 (The Pile)

Max Activating Token Index: 805

Text #3

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.5329. Min Act: -0.0000

Data Index: 25737 (The Pile)

Max Activating Token Index: 570

Text #4

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.4465. Min Act: -0.0000

Data Index: 448487 (The Pile)

Max Activating Token Index: 588

Text #5

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.0829. Min Act: -0.0000

Data Index: 1667061 (The Pile)

Max Activating Token Index: 342

Text #6

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.5184. Min Act: -0.0000

Data Index: 357071 (The Pile)

Max Activating Token Index: 694

Text #7

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.4648. Min Act: -0.0000

Data Index: 616125 (The Pile)

Max Activating Token Index: 820

Text #8

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.4308. Min Act: -0.0000

Data Index: 892290 (The Pile)

Max Activating Token Index: 451

Text #9

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.3867. Min Act: -0.0000

Data Index: 1780876 (The Pile)

Max Activating Token Index: 167

Text #10

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.2886. Min Act: -0.0000

Data Index: 1866114 (The Pile)

Max Activating Token Index: 835

Text #11

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.3857. Min Act: -0.0000

Data Index: 785366 (The Pile)

Max Activating Token Index: 419

Text #12

Max Range: 7.7181. Min Range: -7.7181

Max Act: 6.9787. Min Act: -0.0000

Data Index: 1095672 (The Pile)

Max Activating Token Index: 217

Text #13

Max Range: 7.7181. Min Range: -7.7181

Max Act: 6.9679. Min Act: -0.0000

Data Index: 1718570 (The Pile)

Max Activating Token Index: 636

Text #14

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.1838. Min Act: -0.0000

Data Index: 617851 (The Pile)

Max Activating Token Index: 422

Text #15

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.1588. Min Act: -0.0000

Data Index: 1259185 (The Pile)

Max Activating Token Index: 790

Text #16

Max Range: 7.7181. Min Range: -7.7181

Max Act: 6.7892. Min Act: -0.0000

Data Index: 643020 (The Pile)

Max Activating Token Index: 754

Text #17

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.3691. Min Act: -0.0000

Data Index: 1991879 (The Pile)

Max Activating Token Index: 680

Text #18

Max Range: 7.7181. Min Range: -7.7181

Max Act: 7.1101. Min Act: -0.0000

Data Index: 1694688 (The Pile)

Max Activating Token Index: 442

Text #19

Max Range: 7.7181. Min Range: -7.7181

Max Act: 6.8767. Min Act: -0.0000

Data Index: 1797265 (The Pile)

Max Activating Token Index: 137