Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3598 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1749. Min Range: -1.1749

Max Act: 1.1517. Min Act: -0.0000

Data Index: 1685078 (The Pile)

Max Activating Token Index: 22

Text #1

Max Range: 1.1749. Min Range: -1.1749

Max Act: 1.1749. Min Act: -0.0000

Data Index: 1332733 (The Pile)

Max Activating Token Index: 631

Text #2

Max Range: 1.1749. Min Range: -1.1749

Max Act: 1.0135. Min Act: -0.0000

Data Index: 117152 (The Pile)

Max Activating Token Index: 524

Text #3

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.9297. Min Act: -0.0000

Data Index: 1687310 (The Pile)

Max Activating Token Index: 294

Text #4

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.8979. Min Act: -0.0000

Data Index: 340153 (The Pile)

Max Activating Token Index: 1019

Text #5

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.9252. Min Act: -0.0000

Data Index: 1775749 (The Pile)

Max Activating Token Index: 941

Text #6

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.8845. Min Act: -0.0000

Data Index: 477192 (The Pile)

Max Activating Token Index: 116

Text #7

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.8386. Min Act: -0.0000

Data Index: 1358863 (The Pile)

Max Activating Token Index: 832

Text #8

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.8664. Min Act: -0.0000

Data Index: 762761 (The Pile)

Max Activating Token Index: 629

Text #9

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.8423. Min Act: -0.0000

Data Index: 376406 (The Pile)

Max Activating Token Index: 301

Text #10

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.8316. Min Act: -0.0000

Data Index: 902155 (The Pile)

Max Activating Token Index: 39

Text #11

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.8526. Min Act: -0.0000

Data Index: 1256042 (The Pile)

Max Activating Token Index: 926

Text #12

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.6668. Min Act: -0.0000

Data Index: 840810 (The Pile)

Max Activating Token Index: 876

Text #13

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.8765. Min Act: -0.0000

Data Index: 1283858 (The Pile)

Max Activating Token Index: 989

Text #14

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.8163. Min Act: -0.0000

Data Index: 249007 (The Pile)

Max Activating Token Index: 107

Text #15

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.8099. Min Act: -0.0000

Data Index: 1227567 (The Pile)

Max Activating Token Index: 272

Text #16

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.8294. Min Act: -0.0000

Data Index: 1985729 (The Pile)

Max Activating Token Index: 511

Text #17

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.7805. Min Act: -0.0000

Data Index: 1421773 (The Pile)

Max Activating Token Index: 82

Text #18

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.8464. Min Act: -0.0000

Data Index: 148892 (The Pile)

Max Activating Token Index: 945

Text #19

Max Range: 1.1749. Min Range: -1.1749

Max Act: 0.7750. Min Act: -0.0000

Data Index: 709472 (The Pile)

Max Activating Token Index: 19