Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3685 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.4677. Min Range: -3.4677

Max Act: 3.4677. Min Act: -0.0000

Data Index: 432499 (The Pile)

Max Activating Token Index: 158

Text #1

Max Range: 3.4677. Min Range: -3.4677

Max Act: 3.1632. Min Act: -0.0000

Data Index: 332169 (The Pile)

Max Activating Token Index: 662

Text #2

Max Range: 3.4677. Min Range: -3.4677

Max Act: 3.0176. Min Act: -0.0000

Data Index: 1196400 (The Pile)

Max Activating Token Index: 637

Text #3

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.6927. Min Act: -0.0000

Data Index: 1600860 (The Pile)

Max Activating Token Index: 147

Text #4

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.3782. Min Act: -0.0000

Data Index: 934576 (The Pile)

Max Activating Token Index: 835

Text #5

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.6203. Min Act: -0.0000

Data Index: 1748122 (The Pile)

Max Activating Token Index: 974

Text #6

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.4445. Min Act: -0.0000

Data Index: 1356346 (The Pile)

Max Activating Token Index: 60

Text #7

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.4355. Min Act: -0.0000

Data Index: 119783 (The Pile)

Max Activating Token Index: 124

Text #8

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.2123. Min Act: -0.0000

Data Index: 1562783 (The Pile)

Max Activating Token Index: 141

Text #9

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.4698. Min Act: -0.0000

Data Index: 254158 (The Pile)

Max Activating Token Index: 781

Text #10

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.5019. Min Act: -0.0000

Data Index: 659522 (The Pile)

Max Activating Token Index: 577

Text #11

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.3553. Min Act: -0.0000

Data Index: 81089 (The Pile)

Max Activating Token Index: 810

Text #12

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.2331. Min Act: -0.0000

Data Index: 1311138 (The Pile)

Max Activating Token Index: 791

Text #13

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.1666. Min Act: -0.0000

Data Index: 652466 (The Pile)

Max Activating Token Index: 628

Text #14

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.2631. Min Act: -0.0000

Data Index: 1443099 (The Pile)

Max Activating Token Index: 516

Text #15

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.1205. Min Act: -0.0000

Data Index: 1067390 (The Pile)

Max Activating Token Index: 172

Text #16

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.2073. Min Act: -0.0000

Data Index: 1963591 (The Pile)

Max Activating Token Index: 381

Text #17

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.4909. Min Act: -0.0000

Data Index: 984921 (The Pile)

Max Activating Token Index: 918

Text #18

Max Range: 3.4677. Min Range: -3.4677

Max Act: 1.9839. Min Act: -0.0000

Data Index: 308341 (The Pile)

Max Activating Token Index: 394

Text #19

Max Range: 3.4677. Min Range: -3.4677

Max Act: 2.0108. Min Act: -0.0000

Data Index: 1752632 (The Pile)

Max Activating Token Index: 105