Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 182 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.1742. Min Range: -3.1742

Max Act: 3.1742. Min Act: -0.0000

Data Index: 205772 (The Pile)

Max Activating Token Index: 177

Text #1

Max Range: 3.1742. Min Range: -3.1742

Max Act: 2.4411. Min Act: -0.0000

Data Index: 1329676 (The Pile)

Max Activating Token Index: 291

Text #2

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.8869. Min Act: -0.0000

Data Index: 137482 (The Pile)

Max Activating Token Index: 118

Text #3

Max Range: 3.1742. Min Range: -3.1742

Max Act: 2.0440. Min Act: -0.0000

Data Index: 1821927 (The Pile)

Max Activating Token Index: 215

Text #4

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.8585. Min Act: -0.0000

Data Index: 1994107 (The Pile)

Max Activating Token Index: 558

Text #5

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.7497. Min Act: -0.0000

Data Index: 677989 (The Pile)

Max Activating Token Index: 578

Text #6

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.7004. Min Act: -0.0000

Data Index: 270209 (The Pile)

Max Activating Token Index: 877

Text #7

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.5833. Min Act: -0.0000

Data Index: 699948 (The Pile)

Max Activating Token Index: 156

Text #8

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.4432. Min Act: -0.0000

Data Index: 1029528 (The Pile)

Max Activating Token Index: 30

Text #9

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.4236. Min Act: -0.0000

Data Index: 1634819 (The Pile)

Max Activating Token Index: 79

Text #10

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.4960. Min Act: -0.0000

Data Index: 783549 (The Pile)

Max Activating Token Index: 308

Text #11

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.4396. Min Act: -0.0000

Data Index: 284354 (The Pile)

Max Activating Token Index: 324

Text #12

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.5063. Min Act: -0.0000

Data Index: 928068 (The Pile)

Max Activating Token Index: 72

Text #13

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.3736. Min Act: -0.0000

Data Index: 217456 (The Pile)

Max Activating Token Index: 800

Text #14

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.4589. Min Act: -0.0000

Data Index: 1153308 (The Pile)

Max Activating Token Index: 78

Text #15

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.4005. Min Act: -0.0000

Data Index: 178259 (The Pile)

Max Activating Token Index: 323

Text #16

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.4245. Min Act: -0.0000

Data Index: 986712 (The Pile)

Max Activating Token Index: 82

Text #17

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.4173. Min Act: -0.0000

Data Index: 759750 (The Pile)

Max Activating Token Index: 282

Text #18

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.4067. Min Act: -0.0000

Data Index: 621124 (The Pile)

Max Activating Token Index: 540

Text #19

Max Range: 3.1742. Min Range: -3.1742

Max Act: 1.4121. Min Act: -0.0000

Data Index: 903132 (The Pile)

Max Activating Token Index: 38