Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3683 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0059. Min Range: -1.0059

Max Act: 1.0059. Min Act: -0.0000

Data Index: 726148 (The Pile)

Max Activating Token Index: 12

Text #1

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.6519. Min Act: -0.0000

Data Index: 1249722 (The Pile)

Max Activating Token Index: 63

Text #2

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.6687. Min Act: -0.0000

Data Index: 1170637 (The Pile)

Max Activating Token Index: 7

Text #3

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.6104. Min Act: -0.0000

Data Index: 1317030 (The Pile)

Max Activating Token Index: 565

Text #4

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.5980. Min Act: -0.0000

Data Index: 838606 (The Pile)

Max Activating Token Index: 716

Text #5

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.6375. Min Act: -0.0000

Data Index: 672412 (The Pile)

Max Activating Token Index: 323

Text #6

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.6502. Min Act: -0.0000

Data Index: 995515 (The Pile)

Max Activating Token Index: 52

Text #7

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.5846. Min Act: -0.0000

Data Index: 691737 (The Pile)

Max Activating Token Index: 96

Text #8

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.5607. Min Act: -0.0000

Data Index: 350599 (The Pile)

Max Activating Token Index: 384

Text #9

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.5835. Min Act: -0.0000

Data Index: 1291454 (The Pile)

Max Activating Token Index: 5

Text #10

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.5612. Min Act: -0.0000

Data Index: 1067546 (The Pile)

Max Activating Token Index: 218

Text #11

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.5772. Min Act: -0.0000

Data Index: 1992142 (The Pile)

Max Activating Token Index: 619

Text #12

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.5406. Min Act: -0.0000

Data Index: 1568058 (The Pile)

Max Activating Token Index: 118

Text #13

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.5152. Min Act: -0.0000

Data Index: 1641353 (The Pile)

Max Activating Token Index: 474

Text #14

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.5428. Min Act: -0.0000

Data Index: 1773739 (The Pile)

Max Activating Token Index: 349

Text #15

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.5273. Min Act: -0.0000

Data Index: 669221 (The Pile)

Max Activating Token Index: 405

Text #16

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.6225. Min Act: -0.0000

Data Index: 1659405 (The Pile)

Max Activating Token Index: 849

Text #17

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.5577. Min Act: -0.0000

Data Index: 1015893 (The Pile)

Max Activating Token Index: 431

Text #18

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.6473. Min Act: -0.0000

Data Index: 1203404 (The Pile)

Max Activating Token Index: 847

Text #19

Max Range: 1.0059. Min Range: -1.0059

Max Act: 0.5534. Min Act: -0.0000

Data Index: 954381 (The Pile)

Max Activating Token Index: 111