Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 660 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 2.1109. Min Range: -2.1109

Max Act: 2.0121. Min Act: -0.0000

Data Index: 62317 (The Pile)

Max Activating Token Index: 939

Text #1

Max Range: 2.1109. Min Range: -2.1109

Max Act: 2.1109. Min Act: -0.0000

Data Index: 903277 (The Pile)

Max Activating Token Index: 273

Text #2

Max Range: 2.1109. Min Range: -2.1109

Max Act: 2.0013. Min Act: -0.0000

Data Index: 1819709 (The Pile)

Max Activating Token Index: 794

Text #3

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.9168. Min Act: -0.0000

Data Index: 81500 (The Pile)

Max Activating Token Index: 238

Text #4

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.7765. Min Act: -0.0000

Data Index: 820326 (The Pile)

Max Activating Token Index: 517

Text #5

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.8637. Min Act: -0.0000

Data Index: 152245 (The Pile)

Max Activating Token Index: 723

Text #6

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.7281. Min Act: -0.0000

Data Index: 858254 (The Pile)

Max Activating Token Index: 611

Text #7

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.5514. Min Act: -0.0000

Data Index: 1640845 (The Pile)

Max Activating Token Index: 745

Text #8

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.6153. Min Act: -0.0000

Data Index: 479143 (The Pile)

Max Activating Token Index: 777

Text #9

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.5574. Min Act: -0.0000

Data Index: 1445972 (The Pile)

Max Activating Token Index: 420

Text #10

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.6268. Min Act: -0.0000

Data Index: 541938 (The Pile)

Max Activating Token Index: 383

Text #11

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.6105. Min Act: -0.0000

Data Index: 609833 (The Pile)

Max Activating Token Index: 370

Text #12

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.7102. Min Act: -0.0000

Data Index: 1999934 (The Pile)

Max Activating Token Index: 98

Text #13

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.7151. Min Act: -0.0000

Data Index: 450017 (The Pile)

Max Activating Token Index: 340

Text #14

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.5111. Min Act: -0.0000

Data Index: 1312648 (The Pile)

Max Activating Token Index: 225

Text #15

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.7724. Min Act: -0.0000

Data Index: 1833226 (The Pile)

Max Activating Token Index: 644

Text #16

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.5800. Min Act: -0.0000

Data Index: 963564 (The Pile)

Max Activating Token Index: 678

Text #17

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.6719. Min Act: -0.0000

Data Index: 1343229 (The Pile)

Max Activating Token Index: 993

Text #18

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.4629. Min Act: -0.0000

Data Index: 1415998 (The Pile)

Max Activating Token Index: 534

Text #19

Max Range: 2.1109. Min Range: -2.1109

Max Act: 1.4370. Min Act: -0.0000

Data Index: 1777349 (The Pile)

Max Activating Token Index: 158