Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4645 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.6124. Min Act: -0.0000

Data Index: 659234 (The Pile)

Max Activating Token Index: 752

Text #1

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.6544. Min Act: -0.0000

Data Index: 1313571 (The Pile)

Max Activating Token Index: 952

Text #2

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.5983. Min Act: -0.0000

Data Index: 249841 (The Pile)

Max Activating Token Index: 762

Text #3

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.5789. Min Act: -0.0000

Data Index: 1601607 (The Pile)

Max Activating Token Index: 922

Text #4

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.5178. Min Act: -0.0000

Data Index: 1200287 (The Pile)

Max Activating Token Index: 387

Text #5

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.5438. Min Act: -0.0000

Data Index: 1738431 (The Pile)

Max Activating Token Index: 719

Text #6

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.5095. Min Act: -0.0000

Data Index: 39514 (The Pile)

Max Activating Token Index: 52

Text #7

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.5059. Min Act: -0.0000

Data Index: 162879 (The Pile)

Max Activating Token Index: 662

Text #8

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.4898. Min Act: -0.0000

Data Index: 1841257 (The Pile)

Max Activating Token Index: 49

Text #9

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.4976. Min Act: -0.0000

Data Index: 1280028 (The Pile)

Max Activating Token Index: 345

Text #10

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.4826. Min Act: -0.0000

Data Index: 652485 (The Pile)

Max Activating Token Index: 823

Text #11

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.4928. Min Act: -0.0000

Data Index: 938024 (The Pile)

Max Activating Token Index: 222

Text #12

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.4787. Min Act: -0.0000

Data Index: 767926 (The Pile)

Max Activating Token Index: 553

Text #13

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.4758. Min Act: -0.0000

Data Index: 1626754 (The Pile)

Max Activating Token Index: 428

Text #14

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.4627. Min Act: -0.0000

Data Index: 1981168 (The Pile)

Max Activating Token Index: 29

Text #15

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.5103. Min Act: -0.0000

Data Index: 1862218 (The Pile)

Max Activating Token Index: 167

Text #16

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.5047. Min Act: -0.0000

Data Index: 1299430 (The Pile)

Max Activating Token Index: 732

Text #17

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.4712. Min Act: -0.0000

Data Index: 1527229 (The Pile)

Max Activating Token Index: 49

Text #18

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.4505. Min Act: -0.0000

Data Index: 1754854 (The Pile)

Max Activating Token Index: 715

Text #19

Max Range: 0.6544. Min Range: -0.6544

Max Act: 0.4509. Min Act: -0.0000

Data Index: 969602 (The Pile)

Max Activating Token Index: 293