Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1318 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.3252. Min Act: -0.0000

Data Index: 441032 (The Pile)

Max Activating Token Index: 124

Text #1

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.2308. Min Act: -0.0000

Data Index: 1955507 (The Pile)

Max Activating Token Index: 817

Text #2

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.1923. Min Act: -0.0000

Data Index: 754694 (The Pile)

Max Activating Token Index: 994

Text #3

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.2088. Min Act: -0.0000

Data Index: 474037 (The Pile)

Max Activating Token Index: 252

Text #4

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.1975. Min Act: -0.0000

Data Index: 1217009 (The Pile)

Max Activating Token Index: 75

Text #5

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.2094. Min Act: -0.0000

Data Index: 767764 (The Pile)

Max Activating Token Index: 734

Text #6

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.2412. Min Act: -0.0000

Data Index: 247223 (The Pile)

Max Activating Token Index: 883

Text #7

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.2121. Min Act: -0.0000

Data Index: 485730 (The Pile)

Max Activating Token Index: 851

Text #8

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.2098. Min Act: -0.0000

Data Index: 109936 (The Pile)

Max Activating Token Index: 663

Text #9

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.1848. Min Act: -0.0000

Data Index: 243306 (The Pile)

Max Activating Token Index: 594

Text #10

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.1824. Min Act: -0.0000

Data Index: 1348310 (The Pile)

Max Activating Token Index: 816

Text #11

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.1861. Min Act: -0.0000

Data Index: 1883213 (The Pile)

Max Activating Token Index: 494

Text #12

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.1818. Min Act: -0.0000

Data Index: 1100354 (The Pile)

Max Activating Token Index: 60

Text #13

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.1801. Min Act: -0.0000

Data Index: 1443874 (The Pile)

Max Activating Token Index: 603

Text #14

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.1866. Min Act: -0.0000

Data Index: 681689 (The Pile)

Max Activating Token Index: 282

Text #15

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.2031. Min Act: -0.0000

Data Index: 1861034 (The Pile)

Max Activating Token Index: 825

Text #16

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.1661. Min Act: -0.0000

Data Index: 250564 (The Pile)

Max Activating Token Index: 867

Text #17

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.1768. Min Act: -0.0000

Data Index: 818887 (The Pile)

Max Activating Token Index: 890

Text #18

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.1631. Min Act: -0.0000

Data Index: 1279585 (The Pile)

Max Activating Token Index: 821

Text #19

Max Range: 0.3252. Min Range: -0.3252

Max Act: 0.1621. Min Act: -0.0000

Data Index: 1830151 (The Pile)

Max Activating Token Index: 357