Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1781 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.0889. Min Range: -1.0889

Max Act: 1.0889. Min Act: -0.0000

Data Index: 841453 (The Pile)

Max Activating Token Index: 262

Text #1

Max Range: 1.0889. Min Range: -1.0889

Max Act: 1.0270. Min Act: -0.0000

Data Index: 1210132 (The Pile)

Max Activating Token Index: 630

Text #2

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.9578. Min Act: -0.0000

Data Index: 265421 (The Pile)

Max Activating Token Index: 949

Text #3

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.9790. Min Act: -0.0000

Data Index: 587896 (The Pile)

Max Activating Token Index: 958

Text #4

Max Range: 1.0889. Min Range: -1.0889

Max Act: 1.0028. Min Act: -0.0000

Data Index: 1255866 (The Pile)

Max Activating Token Index: 573

Text #5

Max Range: 1.0889. Min Range: -1.0889

Max Act: 1.0071. Min Act: -0.0000

Data Index: 1620344 (The Pile)

Max Activating Token Index: 556

Text #6

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.9422. Min Act: -0.0000

Data Index: 822893 (The Pile)

Max Activating Token Index: 628

Text #7

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.9330. Min Act: -0.0000

Data Index: 1001637 (The Pile)

Max Activating Token Index: 852

Text #8

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.9431. Min Act: -0.0000

Data Index: 82442 (The Pile)

Max Activating Token Index: 911

Text #9

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.9817. Min Act: -0.0000

Data Index: 10707 (The Pile)

Max Activating Token Index: 480

Text #10

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.9205. Min Act: -0.0000

Data Index: 1841852 (The Pile)

Max Activating Token Index: 968

Text #11

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.9817. Min Act: -0.0000

Data Index: 1695043 (The Pile)

Max Activating Token Index: 375

Text #12

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.8753. Min Act: -0.0000

Data Index: 691024 (The Pile)

Max Activating Token Index: 711

Text #13

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.9052. Min Act: -0.0000

Data Index: 1143327 (The Pile)

Max Activating Token Index: 976

Text #14

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.8925. Min Act: -0.0000

Data Index: 787944 (The Pile)

Max Activating Token Index: 177

Text #15

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.9140. Min Act: -0.0000

Data Index: 1210275 (The Pile)

Max Activating Token Index: 608

Text #16

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.9109. Min Act: -0.0000

Data Index: 1497005 (The Pile)

Max Activating Token Index: 810

Text #17

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.9183. Min Act: -0.0000

Data Index: 25313 (The Pile)

Max Activating Token Index: 943

Text #18

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.8740. Min Act: -0.0000

Data Index: 211933 (The Pile)

Max Activating Token Index: 318

Text #19

Max Range: 1.0889. Min Range: -1.0889

Max Act: 0.8823. Min Act: -0.0000

Data Index: 975462 (The Pile)

Max Activating Token Index: 107