Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4852 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.9525. Min Act: -0.0000

Data Index: 1949032 (The Pile)

Max Activating Token Index: 489

Text #1

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.5786. Min Act: -0.0000

Data Index: 1377546 (The Pile)

Max Activating Token Index: 711

Text #2

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.3841. Min Act: -0.0000

Data Index: 953346 (The Pile)

Max Activating Token Index: 642

Text #3

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.5282. Min Act: -0.0000

Data Index: 1051441 (The Pile)

Max Activating Token Index: 151

Text #4

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.7114. Min Act: -0.0000

Data Index: 721987 (The Pile)

Max Activating Token Index: 656

Text #5

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.2125. Min Act: -0.0000

Data Index: 553121 (The Pile)

Max Activating Token Index: 440

Text #6

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.3474. Min Act: -0.0000

Data Index: 344353 (The Pile)

Max Activating Token Index: 878

Text #7

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.3198. Min Act: -0.0000

Data Index: 1164743 (The Pile)

Max Activating Token Index: 806

Text #8

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.2519. Min Act: -0.0000

Data Index: 1553030 (The Pile)

Max Activating Token Index: 585

Text #9

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.1140. Min Act: -0.0000

Data Index: 845425 (The Pile)

Max Activating Token Index: 641

Text #10

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.2483. Min Act: -0.0000

Data Index: 1607619 (The Pile)

Max Activating Token Index: 1011

Text #11

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.1873. Min Act: -0.0000

Data Index: 1181426 (The Pile)

Max Activating Token Index: 619

Text #12

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.3121. Min Act: -0.0000

Data Index: 528674 (The Pile)

Max Activating Token Index: 773

Text #13

Max Range: 3.9525. Min Range: -3.9525

Max Act: 2.9125. Min Act: -0.0000

Data Index: 181481 (The Pile)

Max Activating Token Index: 868

Text #14

Max Range: 3.9525. Min Range: -3.9525

Max Act: 2.8987. Min Act: -0.0000

Data Index: 245658 (The Pile)

Max Activating Token Index: 140

Text #15

Max Range: 3.9525. Min Range: -3.9525

Max Act: 2.9580. Min Act: -0.0000

Data Index: 1670314 (The Pile)

Max Activating Token Index: 407

Text #16

Max Range: 3.9525. Min Range: -3.9525

Max Act: 2.9466. Min Act: -0.0000

Data Index: 1277719 (The Pile)

Max Activating Token Index: 1021

Text #17

Max Range: 3.9525. Min Range: -3.9525

Max Act: 2.9365. Min Act: -0.0000

Data Index: 497444 (The Pile)

Max Activating Token Index: 680

Text #18

Max Range: 3.9525. Min Range: -3.9525

Max Act: 2.9126. Min Act: -0.0000

Data Index: 388589 (The Pile)

Max Activating Token Index: 456

Text #19

Max Range: 3.9525. Min Range: -3.9525

Max Act: 3.1697. Min Act: -0.0000

Data Index: 1992703 (The Pile)

Max Activating Token Index: 882