Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3888 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.4489. Min Act: -0.0000

Data Index: 7906 (The Pile)

Max Activating Token Index: 184

Text #1

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.4959. Min Act: -0.0000

Data Index: 1676345 (The Pile)

Max Activating Token Index: 611

Text #2

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.6113. Min Act: -0.0000

Data Index: 528184 (The Pile)

Max Activating Token Index: 9

Text #3

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.3947. Min Act: -0.0000

Data Index: 1427578 (The Pile)

Max Activating Token Index: 793

Text #4

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.6543. Min Act: -0.0000

Data Index: 990481 (The Pile)

Max Activating Token Index: 974

Text #5

Max Range: 3.6543. Min Range: -3.6543

Max Act: 2.6680. Min Act: -0.0000

Data Index: 884164 (The Pile)

Max Activating Token Index: 870

Text #6

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.4694. Min Act: -0.0000

Data Index: 1755690 (The Pile)

Max Activating Token Index: 23

Text #7

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.5066. Min Act: -0.0000

Data Index: 43785 (The Pile)

Max Activating Token Index: 792

Text #8

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.4055. Min Act: -0.0000

Data Index: 1126456 (The Pile)

Max Activating Token Index: 90

Text #9

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.2403. Min Act: -0.0000

Data Index: 540550 (The Pile)

Max Activating Token Index: 182

Text #10

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.3995. Min Act: -0.0000

Data Index: 333682 (The Pile)

Max Activating Token Index: 130

Text #11

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.3825. Min Act: -0.0000

Data Index: 896556 (The Pile)

Max Activating Token Index: 689

Text #12

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.0642. Min Act: -0.0000

Data Index: 1483799 (The Pile)

Max Activating Token Index: 142

Text #13

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.5057. Min Act: -0.0000

Data Index: 240992 (The Pile)

Max Activating Token Index: 933

Text #14

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.3294. Min Act: -0.0000

Data Index: 12469 (The Pile)

Max Activating Token Index: 431

Text #15

Max Range: 3.6543. Min Range: -3.6543

Max Act: 2.9697. Min Act: -0.0000

Data Index: 326902 (The Pile)

Max Activating Token Index: 973

Text #16

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.2520. Min Act: -0.0000

Data Index: 1395885 (The Pile)

Max Activating Token Index: 53

Text #17

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.3727. Min Act: -0.0000

Data Index: 1841658 (The Pile)

Max Activating Token Index: 53

Text #18

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.0902. Min Act: -0.0000

Data Index: 1028221 (The Pile)

Max Activating Token Index: 743

Text #19

Max Range: 3.6543. Min Range: -3.6543

Max Act: 3.0733. Min Act: -0.0000

Data Index: 662091 (The Pile)

Max Activating Token Index: 473