Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 773 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.5687. Min Act: -0.0000

Data Index: 191661 (The Pile)

Max Activating Token Index: 501

Text #1

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.5124. Min Act: -0.0000

Data Index: 847977 (The Pile)

Max Activating Token Index: 353

Text #2

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.4833. Min Act: -0.0000

Data Index: 1043848 (The Pile)

Max Activating Token Index: 37

Text #3

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.4681. Min Act: -0.0000

Data Index: 1562780 (The Pile)

Max Activating Token Index: 151

Text #4

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.4965. Min Act: -0.0000

Data Index: 1742147 (The Pile)

Max Activating Token Index: 868

Text #5

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.4182. Min Act: -0.0000

Data Index: 408523 (The Pile)

Max Activating Token Index: 113

Text #6

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.3341. Min Act: -0.0000

Data Index: 412495 (The Pile)

Max Activating Token Index: 718

Text #7

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.2189. Min Act: -0.0000

Data Index: 903676 (The Pile)

Max Activating Token Index: 583

Text #8

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.2686. Min Act: -0.0000

Data Index: 125955 (The Pile)

Max Activating Token Index: 491

Text #9

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.2649. Min Act: -0.0000

Data Index: 1751 (The Pile)

Max Activating Token Index: 141

Text #10

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.2296. Min Act: -0.0000

Data Index: 260915 (The Pile)

Max Activating Token Index: 185

Text #11

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.3222. Min Act: -0.0000

Data Index: 466217 (The Pile)

Max Activating Token Index: 529

Text #12

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.2398. Min Act: -0.0000

Data Index: 1334883 (The Pile)

Max Activating Token Index: 200

Text #13

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.1519. Min Act: -0.0000

Data Index: 1425302 (The Pile)

Max Activating Token Index: 622

Text #14

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.2090. Min Act: -0.0000

Data Index: 1446017 (The Pile)

Max Activating Token Index: 112

Text #15

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.1945. Min Act: -0.0000

Data Index: 354061 (The Pile)

Max Activating Token Index: 291

Text #16

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.1559. Min Act: -0.0000

Data Index: 1086192 (The Pile)

Max Activating Token Index: 591

Text #17

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.1812. Min Act: -0.0000

Data Index: 344568 (The Pile)

Max Activating Token Index: 590

Text #18

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.1705. Min Act: -0.0000

Data Index: 176482 (The Pile)

Max Activating Token Index: 927

Text #19

Max Range: 1.5687. Min Range: -1.5687

Max Act: 1.2027. Min Act: -0.0000

Data Index: 1624817 (The Pile)

Max Activating Token Index: 355