Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4382 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.7847. Min Act: -0.0000

Data Index: 1007524 (The Pile)

Max Activating Token Index: 545

Text #1

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.7127. Min Act: -0.0000

Data Index: 730557 (The Pile)

Max Activating Token Index: 658

Text #2

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.7230. Min Act: -0.0000

Data Index: 713610 (The Pile)

Max Activating Token Index: 60

Text #3

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.7206. Min Act: -0.0000

Data Index: 1178722 (The Pile)

Max Activating Token Index: 211

Text #4

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6923. Min Act: -0.0000

Data Index: 847682 (The Pile)

Max Activating Token Index: 778

Text #5

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.7077. Min Act: -0.0000

Data Index: 1166478 (The Pile)

Max Activating Token Index: 55

Text #6

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.7149. Min Act: -0.0000

Data Index: 583454 (The Pile)

Max Activating Token Index: 192

Text #7

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6849. Min Act: -0.0000

Data Index: 1511477 (The Pile)

Max Activating Token Index: 241

Text #8

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6754. Min Act: -0.0000

Data Index: 1747483 (The Pile)

Max Activating Token Index: 178

Text #9

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6861. Min Act: -0.0000

Data Index: 760615 (The Pile)

Max Activating Token Index: 1023

Text #10

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6734. Min Act: -0.0000

Data Index: 72506 (The Pile)

Max Activating Token Index: 720

Text #11

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6955. Min Act: -0.0000

Data Index: 1286577 (The Pile)

Max Activating Token Index: 994

Text #12

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.7209. Min Act: -0.0000

Data Index: 1365936 (The Pile)

Max Activating Token Index: 717

Text #13

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6114. Min Act: -0.0000

Data Index: 329974 (The Pile)

Max Activating Token Index: 878

Text #14

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6623. Min Act: -0.0000

Data Index: 151866 (The Pile)

Max Activating Token Index: 145

Text #15

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6832. Min Act: -0.0000

Data Index: 1785292 (The Pile)

Max Activating Token Index: 322

Text #16

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6793. Min Act: -0.0000

Data Index: 346143 (The Pile)

Max Activating Token Index: 642

Text #17

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6909. Min Act: -0.0000

Data Index: 1895669 (The Pile)

Max Activating Token Index: 36

Text #18

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6781. Min Act: -0.0000

Data Index: 1047678 (The Pile)

Max Activating Token Index: 460

Text #19

Max Range: 0.7847. Min Range: -0.7847

Max Act: 0.6759. Min Act: -0.0000

Data Index: 481605 (The Pile)

Max Activating Token Index: 657