Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2808 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.7746. Min Act: -0.0000

Data Index: 661376 (The Pile)

Max Activating Token Index: 189

Text #1

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.7470. Min Act: -0.0000

Data Index: 220088 (The Pile)

Max Activating Token Index: 42

Text #2

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5774. Min Act: -0.0000

Data Index: 1062810 (The Pile)

Max Activating Token Index: 992

Text #3

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5167. Min Act: -0.0000

Data Index: 85842 (The Pile)

Max Activating Token Index: 978

Text #4

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5570. Min Act: -0.0000

Data Index: 604114 (The Pile)

Max Activating Token Index: 79

Text #5

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5732. Min Act: -0.0000

Data Index: 1115062 (The Pile)

Max Activating Token Index: 79

Text #6

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5412. Min Act: -0.0000

Data Index: 1445883 (The Pile)

Max Activating Token Index: 269

Text #7

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5519. Min Act: -0.0000

Data Index: 1559303 (The Pile)

Max Activating Token Index: 152

Text #8

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5404. Min Act: -0.0000

Data Index: 641437 (The Pile)

Max Activating Token Index: 663

Text #9

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5257. Min Act: -0.0000

Data Index: 1628640 (The Pile)

Max Activating Token Index: 703

Text #10

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.4876. Min Act: -0.0000

Data Index: 453542 (The Pile)

Max Activating Token Index: 685

Text #11

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5052. Min Act: -0.0000

Data Index: 1875693 (The Pile)

Max Activating Token Index: 202

Text #12

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5763. Min Act: -0.0000

Data Index: 1757392 (The Pile)

Max Activating Token Index: 647

Text #13

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5281. Min Act: -0.0000

Data Index: 1696783 (The Pile)

Max Activating Token Index: 301

Text #14

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5240. Min Act: -0.0000

Data Index: 408379 (The Pile)

Max Activating Token Index: 72

Text #15

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5022. Min Act: -0.0000

Data Index: 887707 (The Pile)

Max Activating Token Index: 294

Text #16

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5169. Min Act: -0.0000

Data Index: 1169161 (The Pile)

Max Activating Token Index: 234

Text #17

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.4951. Min Act: -0.0000

Data Index: 1622719 (The Pile)

Max Activating Token Index: 539

Text #18

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.5310. Min Act: -0.0000

Data Index: 977721 (The Pile)

Max Activating Token Index: 616

Text #19

Max Range: 0.7746. Min Range: -0.7746

Max Act: 0.4533. Min Act: -0.0000

Data Index: 1968594 (The Pile)

Max Activating Token Index: 108