Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4985 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.1016. Min Act: -0.0002

Data Index: 1259265 (The Pile)

Max Activating Token Index: 564

Text #1

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.1001. Min Act: -0.0002

Data Index: 1583126 (The Pile)

Max Activating Token Index: 413

Text #2

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.1005. Min Act: -0.0002

Data Index: 368110 (The Pile)

Max Activating Token Index: 660

Text #3

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0950. Min Act: -0.0002

Data Index: 1961273 (The Pile)

Max Activating Token Index: 442

Text #4

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0970. Min Act: -0.0002

Data Index: 1920079 (The Pile)

Max Activating Token Index: 954

Text #5

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0947. Min Act: -0.0002

Data Index: 1443129 (The Pile)

Max Activating Token Index: 1013

Text #6

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0900. Min Act: -0.0002

Data Index: 922939 (The Pile)

Max Activating Token Index: 1020

Text #7

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0938. Min Act: -0.0002

Data Index: 1516671 (The Pile)

Max Activating Token Index: 356

Text #8

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0931. Min Act: -0.0002

Data Index: 1875667 (The Pile)

Max Activating Token Index: 382

Text #9

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0871. Min Act: -0.0002

Data Index: 107810 (The Pile)

Max Activating Token Index: 414

Text #10

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0885. Min Act: -0.0002

Data Index: 1088519 (The Pile)

Max Activating Token Index: 271

Text #11

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0928. Min Act: -0.0002

Data Index: 1492084 (The Pile)

Max Activating Token Index: 493

Text #12

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0882. Min Act: -0.0002

Data Index: 254162 (The Pile)

Max Activating Token Index: 210

Text #13

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0837. Min Act: -0.0002

Data Index: 1795973 (The Pile)

Max Activating Token Index: 450

Text #14

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0908. Min Act: -0.0002

Data Index: 928247 (The Pile)

Max Activating Token Index: 766

Text #15

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0896. Min Act: -0.0002

Data Index: 635459 (The Pile)

Max Activating Token Index: 374

Text #16

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0854. Min Act: -0.0002

Data Index: 1765759 (The Pile)

Max Activating Token Index: 268

Text #17

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0817. Min Act: -0.0002

Data Index: 73693 (The Pile)

Max Activating Token Index: 619

Text #18

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0829. Min Act: -0.0002

Data Index: 1609655 (The Pile)

Max Activating Token Index: 870

Text #19

Max Range: 0.1016. Min Range: -0.1016

Max Act: 0.0823. Min Act: -0.0002

Data Index: 1906077 (The Pile)

Max Activating Token Index: 885