Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3352 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0753. Min Range: -1.0753

Max Act: 1.0753. Min Act: -0.0000

Data Index: 1887434 (The Pile)

Max Activating Token Index: 293

Text #1

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.8989. Min Act: -0.0000

Data Index: 1618062 (The Pile)

Max Activating Token Index: 687

Text #2

Max Range: 1.0753. Min Range: -1.0753

Max Act: 1.0228. Min Act: -0.0000

Data Index: 1414293 (The Pile)

Max Activating Token Index: 203

Text #3

Max Range: 1.0753. Min Range: -1.0753

Max Act: 1.0321. Min Act: -0.0000

Data Index: 1281264 (The Pile)

Max Activating Token Index: 692

Text #4

Max Range: 1.0753. Min Range: -1.0753

Max Act: 1.0100. Min Act: -0.0000

Data Index: 1728559 (The Pile)

Max Activating Token Index: 340

Text #5

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.8767. Min Act: -0.0000

Data Index: 1436323 (The Pile)

Max Activating Token Index: 17

Text #6

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.9258. Min Act: -0.0000

Data Index: 456146 (The Pile)

Max Activating Token Index: 747

Text #7

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.8993. Min Act: -0.0000

Data Index: 1057694 (The Pile)

Max Activating Token Index: 316

Text #8

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.9551. Min Act: -0.0000

Data Index: 1215484 (The Pile)

Max Activating Token Index: 167

Text #9

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.8423. Min Act: -0.0000

Data Index: 91935 (The Pile)

Max Activating Token Index: 120

Text #10

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.8876. Min Act: -0.0000

Data Index: 493553 (The Pile)

Max Activating Token Index: 381

Text #11

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.8260. Min Act: -0.0000

Data Index: 931908 (The Pile)

Max Activating Token Index: 324

Text #12

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.8644. Min Act: -0.0000

Data Index: 1010854 (The Pile)

Max Activating Token Index: 499

Text #13

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.8089. Min Act: -0.0000

Data Index: 1681849 (The Pile)

Max Activating Token Index: 358

Text #14

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.8224. Min Act: -0.0000

Data Index: 284843 (The Pile)

Max Activating Token Index: 693

Text #15

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.8949. Min Act: -0.0000

Data Index: 487446 (The Pile)

Max Activating Token Index: 890

Text #16

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.8559. Min Act: -0.0000

Data Index: 1404096 (The Pile)

Max Activating Token Index: 532

Text #17

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.7633. Min Act: -0.0000

Data Index: 956197 (The Pile)

Max Activating Token Index: 1016

Text #18

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.8040. Min Act: -0.0000

Data Index: 913249 (The Pile)

Max Activating Token Index: 39

Text #19

Max Range: 1.0753. Min Range: -1.0753

Max Act: 0.7893. Min Act: -0.0000

Data Index: 1081280 (The Pile)

Max Activating Token Index: 598