Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 962 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.4077. Min Act: -0.0000

Data Index: 882966 (The Pile)

Max Activating Token Index: 971

Text #1

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.4003. Min Act: -0.0000

Data Index: 31671 (The Pile)

Max Activating Token Index: 972

Text #2

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.3720. Min Act: -0.0000

Data Index: 576703 (The Pile)

Max Activating Token Index: 712

Text #3

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.3180. Min Act: -0.0000

Data Index: 481533 (The Pile)

Max Activating Token Index: 114

Text #4

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.3079. Min Act: -0.0000

Data Index: 669830 (The Pile)

Max Activating Token Index: 197

Text #5

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.2605. Min Act: -0.0000

Data Index: 603902 (The Pile)

Max Activating Token Index: 634

Text #6

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.2614. Min Act: -0.0000

Data Index: 1416248 (The Pile)

Max Activating Token Index: 118

Text #7

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.2264. Min Act: -0.0000

Data Index: 767326 (The Pile)

Max Activating Token Index: 663

Text #8

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.2151. Min Act: -0.0000

Data Index: 1044695 (The Pile)

Max Activating Token Index: 486

Text #9

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.2163. Min Act: -0.0000

Data Index: 45720 (The Pile)

Max Activating Token Index: 543

Text #10

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.1015. Min Act: -0.0000

Data Index: 690057 (The Pile)

Max Activating Token Index: 605

Text #11

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.2004. Min Act: -0.0000

Data Index: 811820 (The Pile)

Max Activating Token Index: 300

Text #12

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.0530. Min Act: -0.0000

Data Index: 1911114 (The Pile)

Max Activating Token Index: 775

Text #13

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.1596. Min Act: -0.0000

Data Index: 1692563 (The Pile)

Max Activating Token Index: 822

Text #14

Max Range: 1.4077. Min Range: -1.4077

Max Act: 0.9798. Min Act: -0.0000

Data Index: 803841 (The Pile)

Max Activating Token Index: 727

Text #15

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.1281. Min Act: -0.0000

Data Index: 1666805 (The Pile)

Max Activating Token Index: 728

Text #16

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.0511. Min Act: -0.0000

Data Index: 834082 (The Pile)

Max Activating Token Index: 620

Text #17

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.0933. Min Act: -0.0000

Data Index: 621852 (The Pile)

Max Activating Token Index: 378

Text #18

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.0097. Min Act: -0.0000

Data Index: 1957914 (The Pile)

Max Activating Token Index: 318

Text #19

Max Range: 1.4077. Min Range: -1.4077

Max Act: 1.0733. Min Act: -0.0000

Data Index: 318073 (The Pile)

Max Activating Token Index: 485