Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2927 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8060. Min Range: -1.8060

Max Act: 1.8060. Min Act: -0.0000

Data Index: 458700 (The Pile)

Max Activating Token Index: 39

Text #1

Max Range: 1.8060. Min Range: -1.8060

Max Act: 1.2019. Min Act: -0.0000

Data Index: 1104552 (The Pile)

Max Activating Token Index: 254

Text #2

Max Range: 1.8060. Min Range: -1.8060

Max Act: 1.2001. Min Act: -0.0000

Data Index: 94813 (The Pile)

Max Activating Token Index: 540

Text #3

Max Range: 1.8060. Min Range: -1.8060

Max Act: 1.3069. Min Act: -0.0000

Data Index: 1768431 (The Pile)

Max Activating Token Index: 404

Text #4

Max Range: 1.8060. Min Range: -1.8060

Max Act: 1.2182. Min Act: -0.0000

Data Index: 488633 (The Pile)

Max Activating Token Index: 322

Text #5

Max Range: 1.8060. Min Range: -1.8060

Max Act: 1.1596. Min Act: -0.0000

Data Index: 1459935 (The Pile)

Max Activating Token Index: 332

Text #6

Max Range: 1.8060. Min Range: -1.8060

Max Act: 1.0281. Min Act: -0.0000

Data Index: 1264432 (The Pile)

Max Activating Token Index: 631

Text #7

Max Range: 1.8060. Min Range: -1.8060

Max Act: 1.0545. Min Act: -0.0000

Data Index: 611189 (The Pile)

Max Activating Token Index: 503

Text #8

Max Range: 1.8060. Min Range: -1.8060

Max Act: 0.9142. Min Act: -0.0000

Data Index: 1410628 (The Pile)

Max Activating Token Index: 470

Text #9

Max Range: 1.8060. Min Range: -1.8060

Max Act: 1.0305. Min Act: -0.0000

Data Index: 963299 (The Pile)

Max Activating Token Index: 743

Text #10

Max Range: 1.8060. Min Range: -1.8060

Max Act: 0.8532. Min Act: -0.0000

Data Index: 1044460 (The Pile)

Max Activating Token Index: 740

Text #11

Max Range: 1.8060. Min Range: -1.8060

Max Act: 0.8811. Min Act: -0.0000

Data Index: 628349 (The Pile)

Max Activating Token Index: 332

Text #12

Max Range: 1.8060. Min Range: -1.8060

Max Act: 0.8884. Min Act: -0.0000

Data Index: 1375741 (The Pile)

Max Activating Token Index: 800

Text #13

Max Range: 1.8060. Min Range: -1.8060

Max Act: 0.8026. Min Act: -0.0000

Data Index: 448782 (The Pile)

Max Activating Token Index: 346

Text #14

Max Range: 1.8060. Min Range: -1.8060

Max Act: 0.7977. Min Act: -0.0000

Data Index: 1753215 (The Pile)

Max Activating Token Index: 531

Text #15

Max Range: 1.8060. Min Range: -1.8060

Max Act: 0.7764. Min Act: -0.0000

Data Index: 1338523 (The Pile)

Max Activating Token Index: 19

Text #16

Max Range: 1.8060. Min Range: -1.8060

Max Act: 0.7937. Min Act: -0.0000

Data Index: 1401825 (The Pile)

Max Activating Token Index: 140

Text #17

Max Range: 1.8060. Min Range: -1.8060

Max Act: 0.7740. Min Act: -0.0000

Data Index: 231723 (The Pile)

Max Activating Token Index: 918

Text #18

Max Range: 1.8060. Min Range: -1.8060

Max Act: 0.6912. Min Act: -0.0000

Data Index: 1517568 (The Pile)

Max Activating Token Index: 61

Text #19

Max Range: 1.8060. Min Range: -1.8060

Max Act: 0.8096. Min Act: -0.0000

Data Index: 1016816 (The Pile)

Max Activating Token Index: 42