Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3161 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.5623. Min Act: -0.0000

Data Index: 1635658 (The Pile)

Max Activating Token Index: 292

Text #1

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.2967. Min Act: -0.0000

Data Index: 1768202 (The Pile)

Max Activating Token Index: 93

Text #2

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.2215. Min Act: -0.0000

Data Index: 1048030 (The Pile)

Max Activating Token Index: 844

Text #3

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.1516. Min Act: -0.0000

Data Index: 528812 (The Pile)

Max Activating Token Index: 481

Text #4

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.0758. Min Act: -0.0000

Data Index: 1054666 (The Pile)

Max Activating Token Index: 834

Text #5

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.1404. Min Act: -0.0000

Data Index: 1230574 (The Pile)

Max Activating Token Index: 488

Text #6

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.1147. Min Act: -0.0000

Data Index: 1245111 (The Pile)

Max Activating Token Index: 262

Text #7

Max Range: 1.5623. Min Range: -1.5623

Max Act: 0.9633. Min Act: -0.0000

Data Index: 1378249 (The Pile)

Max Activating Token Index: 662

Text #8

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.0827. Min Act: -0.0000

Data Index: 605373 (The Pile)

Max Activating Token Index: 896

Text #9

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.0527. Min Act: -0.0000

Data Index: 413838 (The Pile)

Max Activating Token Index: 999

Text #10

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.0949. Min Act: -0.0000

Data Index: 1585443 (The Pile)

Max Activating Token Index: 796

Text #11

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.0646. Min Act: -0.0000

Data Index: 286865 (The Pile)

Max Activating Token Index: 978

Text #12

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.1038. Min Act: -0.0000

Data Index: 203168 (The Pile)

Max Activating Token Index: 320

Text #13

Max Range: 1.5623. Min Range: -1.5623

Max Act: 0.9979. Min Act: -0.0000

Data Index: 1713196 (The Pile)

Max Activating Token Index: 889

Text #14

Max Range: 1.5623. Min Range: -1.5623

Max Act: 1.0244. Min Act: -0.0000

Data Index: 1189951 (The Pile)

Max Activating Token Index: 485

Text #15

Max Range: 1.5623. Min Range: -1.5623

Max Act: 0.9164. Min Act: -0.0000

Data Index: 1229822 (The Pile)

Max Activating Token Index: 99

Text #16

Max Range: 1.5623. Min Range: -1.5623

Max Act: 0.9720. Min Act: -0.0000

Data Index: 330965 (The Pile)

Max Activating Token Index: 730

Text #17

Max Range: 1.5623. Min Range: -1.5623

Max Act: 0.9828. Min Act: -0.0000

Data Index: 335223 (The Pile)

Max Activating Token Index: 716

Text #18

Max Range: 1.5623. Min Range: -1.5623

Max Act: 0.8599. Min Act: -0.0000

Data Index: 211193 (The Pile)

Max Activating Token Index: 700

Text #19

Max Range: 1.5623. Min Range: -1.5623

Max Act: 0.8944. Min Act: -0.0000

Data Index: 83948 (The Pile)

Max Activating Token Index: 322