Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2945 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.1237. Min Act: -0.0000

Data Index: 1906835 (The Pile)

Max Activating Token Index: 392

Text #1

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.1113. Min Act: -0.0000

Data Index: 1809884 (The Pile)

Max Activating Token Index: 251

Text #2

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.1232. Min Act: -0.0000

Data Index: 1326993 (The Pile)

Max Activating Token Index: 332

Text #3

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.1226. Min Act: -0.0000

Data Index: 101951 (The Pile)

Max Activating Token Index: 411

Text #4

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.1046. Min Act: -0.0000

Data Index: 1188992 (The Pile)

Max Activating Token Index: 345

Text #5

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.1089. Min Act: -0.0000

Data Index: 618779 (The Pile)

Max Activating Token Index: 482

Text #6

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.1096. Min Act: -0.0000

Data Index: 1084181 (The Pile)

Max Activating Token Index: 201

Text #7

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.0943. Min Act: -0.0000

Data Index: 6985 (The Pile)

Max Activating Token Index: 678

Text #8

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.1002. Min Act: -0.0000

Data Index: 1476277 (The Pile)

Max Activating Token Index: 566

Text #9

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.0949. Min Act: -0.0000

Data Index: 1954081 (The Pile)

Max Activating Token Index: 771

Text #10

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.1097. Min Act: -0.0000

Data Index: 1237011 (The Pile)

Max Activating Token Index: 357

Text #11

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.0847. Min Act: -0.0000

Data Index: 1805180 (The Pile)

Max Activating Token Index: 240

Text #12

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.1003. Min Act: -0.0000

Data Index: 313957 (The Pile)

Max Activating Token Index: 439

Text #13

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.0923. Min Act: -0.0000

Data Index: 457175 (The Pile)

Max Activating Token Index: 769

Text #14

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.0935. Min Act: -0.0000

Data Index: 750466 (The Pile)

Max Activating Token Index: 517

Text #15

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.0772. Min Act: -0.0000

Data Index: 781621 (The Pile)

Max Activating Token Index: 1006

Text #16

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.0946. Min Act: -0.0000

Data Index: 1147193 (The Pile)

Max Activating Token Index: 526

Text #17

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.0882. Min Act: -0.0000

Data Index: 843449 (The Pile)

Max Activating Token Index: 515

Text #18

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.1120. Min Act: -0.0000

Data Index: 320973 (The Pile)

Max Activating Token Index: 237

Text #19

Max Range: 0.1237. Min Range: -0.1237

Max Act: 0.0888. Min Act: -0.0000

Data Index: 878041 (The Pile)

Max Activating Token Index: 308