Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4709 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.1016. Min Act: -0.0002

Data Index: 415263 (The Pile)

Max Activating Token Index: 331

Text #1

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0957. Min Act: -0.0002

Data Index: 152705 (The Pile)

Max Activating Token Index: 713

Text #2

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0899. Min Act: -0.0002

Data Index: 1274671 (The Pile)

Max Activating Token Index: 151

Text #3

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0856. Min Act: -0.0002

Data Index: 1804360 (The Pile)

Max Activating Token Index: 613

Text #4

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0890. Min Act: -0.0002

Data Index: 783719 (The Pile)

Max Activating Token Index: 781

Text #5

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0897. Min Act: -0.0002

Data Index: 241939 (The Pile)

Max Activating Token Index: 925

Text #6

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0790. Min Act: -0.0002

Data Index: 580983 (The Pile)

Max Activating Token Index: 51

Text #7

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0751. Min Act: -0.0002

Data Index: 414966 (The Pile)

Max Activating Token Index: 598

Text #8

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0754. Min Act: -0.0002

Data Index: 1552894 (The Pile)

Max Activating Token Index: 674

Text #9

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0778. Min Act: -0.0002

Data Index: 103958 (The Pile)

Max Activating Token Index: 1002

Text #10

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0721. Min Act: -0.0002

Data Index: 1709565 (The Pile)

Max Activating Token Index: 494

Text #11

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0754. Min Act: -0.0002

Data Index: 911807 (The Pile)

Max Activating Token Index: 567

Text #12

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0717. Min Act: -0.0002

Data Index: 1243173 (The Pile)

Max Activating Token Index: 873

Text #13

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0741. Min Act: -0.0002

Data Index: 1353713 (The Pile)

Max Activating Token Index: 671

Text #14

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0719. Min Act: -0.0002

Data Index: 930176 (The Pile)

Max Activating Token Index: 548

Text #15

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0667. Min Act: -0.0002

Data Index: 673917 (The Pile)

Max Activating Token Index: 548

Text #16

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0700. Min Act: -0.0002

Data Index: 708260 (The Pile)

Max Activating Token Index: 255

Text #17

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0664. Min Act: -0.0002

Data Index: 1363536 (The Pile)

Max Activating Token Index: 267

Text #18

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0653. Min Act: -0.0002

Data Index: 701163 (The Pile)

Max Activating Token Index: 86

Text #19

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0623. Min Act: -0.0002

Data Index: 1361353 (The Pile)

Max Activating Token Index: 672