Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2469 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2968. Min Range: -1.2968

Max Act: 1.2968. Min Act: -0.0000

Data Index: 329744 (The Pile)

Max Activating Token Index: 1022

Text #1

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.9686. Min Act: -0.0000

Data Index: 606537 (The Pile)

Max Activating Token Index: 704

Text #2

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.6985. Min Act: -0.0000

Data Index: 1943406 (The Pile)

Max Activating Token Index: 809

Text #3

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.9023. Min Act: -0.0000

Data Index: 780260 (The Pile)

Max Activating Token Index: 761

Text #4

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.8186. Min Act: -0.0000

Data Index: 715387 (The Pile)

Max Activating Token Index: 176

Text #5

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.8328. Min Act: -0.0000

Data Index: 579258 (The Pile)

Max Activating Token Index: 440

Text #6

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.8363. Min Act: -0.0000

Data Index: 655228 (The Pile)

Max Activating Token Index: 529

Text #7

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.8545. Min Act: -0.0000

Data Index: 945965 (The Pile)

Max Activating Token Index: 146

Text #8

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.7899. Min Act: -0.0000

Data Index: 390409 (The Pile)

Max Activating Token Index: 958

Text #9

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.8312. Min Act: -0.0000

Data Index: 1446930 (The Pile)

Max Activating Token Index: 584

Text #10

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.8072. Min Act: -0.0000

Data Index: 525548 (The Pile)

Max Activating Token Index: 539

Text #11

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.7291. Min Act: -0.0000

Data Index: 1155753 (The Pile)

Max Activating Token Index: 283

Text #12

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.9076. Min Act: -0.0000

Data Index: 807909 (The Pile)

Max Activating Token Index: 764

Text #13

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.6215. Min Act: -0.0000

Data Index: 726318 (The Pile)

Max Activating Token Index: 831

Text #14

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.8047. Min Act: -0.0000

Data Index: 1018102 (The Pile)

Max Activating Token Index: 460

Text #15

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.7369. Min Act: -0.0000

Data Index: 1684470 (The Pile)

Max Activating Token Index: 135

Text #16

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.7178. Min Act: -0.0000

Data Index: 429296 (The Pile)

Max Activating Token Index: 445

Text #17

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.6877. Min Act: -0.0000

Data Index: 132359 (The Pile)

Max Activating Token Index: 391

Text #18

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.7133. Min Act: -0.0000

Data Index: 564369 (The Pile)

Max Activating Token Index: 370

Text #19

Max Range: 1.2968. Min Range: -1.2968

Max Act: 0.7219. Min Act: -0.0000

Data Index: 1850202 (The Pile)

Max Activating Token Index: 55