Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3092 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.8205. Min Act: -0.0000

Data Index: 224805 (The Pile)

Max Activating Token Index: 783

Text #1

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.5236. Min Act: -0.0000

Data Index: 564777 (The Pile)

Max Activating Token Index: 24

Text #2

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.5089. Min Act: -0.0000

Data Index: 655499 (The Pile)

Max Activating Token Index: 343

Text #3

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.4828. Min Act: -0.0000

Data Index: 417174 (The Pile)

Max Activating Token Index: 51

Text #4

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.5159. Min Act: -0.0000

Data Index: 783481 (The Pile)

Max Activating Token Index: 15

Text #5

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.4761. Min Act: -0.0000

Data Index: 1658318 (The Pile)

Max Activating Token Index: 91

Text #6

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.5107. Min Act: -0.0000

Data Index: 384136 (The Pile)

Max Activating Token Index: 71

Text #7

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.4680. Min Act: -0.0000

Data Index: 1416269 (The Pile)

Max Activating Token Index: 17

Text #8

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.5129. Min Act: -0.0000

Data Index: 866503 (The Pile)

Max Activating Token Index: 120

Text #9

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.5005. Min Act: -0.0000

Data Index: 1884493 (The Pile)

Max Activating Token Index: 640

Text #10

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.4757. Min Act: -0.0000

Data Index: 83526 (The Pile)

Max Activating Token Index: 1018

Text #11

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.4747. Min Act: -0.0000

Data Index: 1838284 (The Pile)

Max Activating Token Index: 22

Text #12

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.4032. Min Act: -0.0000

Data Index: 205825 (The Pile)

Max Activating Token Index: 739

Text #13

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.4024. Min Act: -0.0000

Data Index: 1896665 (The Pile)

Max Activating Token Index: 185

Text #14

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.3785. Min Act: -0.0000

Data Index: 258700 (The Pile)

Max Activating Token Index: 19

Text #15

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.4281. Min Act: -0.0000

Data Index: 1335778 (The Pile)

Max Activating Token Index: 241

Text #16

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.4197. Min Act: -0.0000

Data Index: 299858 (The Pile)

Max Activating Token Index: 404

Text #17

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.4086. Min Act: -0.0000

Data Index: 1152815 (The Pile)

Max Activating Token Index: 471

Text #18

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.4506. Min Act: -0.0000

Data Index: 1729565 (The Pile)

Max Activating Token Index: 64

Text #19

Max Range: 0.8205. Min Range: -0.8205

Max Act: 0.4161. Min Act: -0.0000

Data Index: 978079 (The Pile)

Max Activating Token Index: 380