Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3515 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0017. Min Act: -0.0000

Data Index: 379573 (The Pile)

Max Activating Token Index: 37

Text #1

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0016. Min Act: -0.0000

Data Index: 894288 (The Pile)

Max Activating Token Index: 83

Text #2

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0016. Min Act: -0.0000

Data Index: 1625915 (The Pile)

Max Activating Token Index: 60

Text #3

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0015. Min Act: -0.0000

Data Index: 361263 (The Pile)

Max Activating Token Index: 35

Text #4

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0015. Min Act: -0.0000

Data Index: 1019148 (The Pile)

Max Activating Token Index: 26

Text #5

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0014. Min Act: -0.0000

Data Index: 1892472 (The Pile)

Max Activating Token Index: 41

Text #6

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0014. Min Act: -0.0000

Data Index: 1571421 (The Pile)

Max Activating Token Index: 2

Text #7

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0014. Min Act: -0.0000

Data Index: 411714 (The Pile)

Max Activating Token Index: 2

Text #8

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0014. Min Act: -0.0000

Data Index: 159694 (The Pile)

Max Activating Token Index: 2

Text #9

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0014. Min Act: -0.0000

Data Index: 105672 (The Pile)

Max Activating Token Index: 30

Text #10

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0014. Min Act: -0.0000

Data Index: 323096 (The Pile)

Max Activating Token Index: 14

Text #11

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0014. Min Act: -0.0000

Data Index: 690358 (The Pile)

Max Activating Token Index: 47

Text #12

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0013. Min Act: -0.0000

Data Index: 1682009 (The Pile)

Max Activating Token Index: 32

Text #13

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0013. Min Act: -0.0000

Data Index: 836782 (The Pile)

Max Activating Token Index: 683

Text #14

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0013. Min Act: -0.0000

Data Index: 1694247 (The Pile)

Max Activating Token Index: 23

Text #15

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0013. Min Act: -0.0000

Data Index: 1803208 (The Pile)

Max Activating Token Index: 51

Text #16

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0013. Min Act: -0.0000

Data Index: 1675110 (The Pile)

Max Activating Token Index: 29

Text #17

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0013. Min Act: -0.0000

Data Index: 1705716 (The Pile)

Max Activating Token Index: 11

Text #18

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0013. Min Act: -0.0000

Data Index: 1869808 (The Pile)

Max Activating Token Index: 59

Text #19

Max Range: 0.0017. Min Range: -0.0017

Max Act: 0.0013. Min Act: -0.0000

Data Index: 30237 (The Pile)

Max Activating Token Index: 18