Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4662 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.5086. Min Act: -0.0000

Data Index: 1519157 (The Pile)

Max Activating Token Index: 817

Text #1

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.4470. Min Act: -0.0000

Data Index: 1276979 (The Pile)

Max Activating Token Index: 363

Text #2

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.4481. Min Act: -0.0000

Data Index: 644043 (The Pile)

Max Activating Token Index: 424

Text #3

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.4017. Min Act: -0.0000

Data Index: 894359 (The Pile)

Max Activating Token Index: 985

Text #4

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.4506. Min Act: -0.0000

Data Index: 1517697 (The Pile)

Max Activating Token Index: 825

Text #5

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.4527. Min Act: -0.0000

Data Index: 11114 (The Pile)

Max Activating Token Index: 759

Text #6

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3831. Min Act: -0.0000

Data Index: 460828 (The Pile)

Max Activating Token Index: 503

Text #7

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3961. Min Act: -0.0000

Data Index: 1149978 (The Pile)

Max Activating Token Index: 894

Text #8

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3855. Min Act: -0.0000

Data Index: 1271863 (The Pile)

Max Activating Token Index: 481

Text #9

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3657. Min Act: -0.0000

Data Index: 1422729 (The Pile)

Max Activating Token Index: 978

Text #10

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3789. Min Act: -0.0000

Data Index: 1893760 (The Pile)

Max Activating Token Index: 247

Text #11

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3124. Min Act: -0.0000

Data Index: 1280259 (The Pile)

Max Activating Token Index: 607

Text #12

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3507. Min Act: -0.0000

Data Index: 946373 (The Pile)

Max Activating Token Index: 88

Text #13

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3128. Min Act: -0.0000

Data Index: 688659 (The Pile)

Max Activating Token Index: 774

Text #14

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3732. Min Act: -0.0000

Data Index: 852984 (The Pile)

Max Activating Token Index: 731

Text #15

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3445. Min Act: -0.0000

Data Index: 189180 (The Pile)

Max Activating Token Index: 815

Text #16

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3460. Min Act: -0.0000

Data Index: 1743000 (The Pile)

Max Activating Token Index: 677

Text #17

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3331. Min Act: -0.0000

Data Index: 578505 (The Pile)

Max Activating Token Index: 503

Text #18

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3424. Min Act: -0.0000

Data Index: 574400 (The Pile)

Max Activating Token Index: 158

Text #19

Max Range: 0.5086. Min Range: -0.5086

Max Act: 0.3192. Min Act: -0.0000

Data Index: 979121 (The Pile)

Max Activating Token Index: 688