Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3668 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.9302. Min Act: -0.0000

Data Index: 607343 (The Pile)

Max Activating Token Index: 548

Text #1

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.9169. Min Act: -0.0000

Data Index: 374696 (The Pile)

Max Activating Token Index: 364

Text #2

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.8689. Min Act: -0.0000

Data Index: 934472 (The Pile)

Max Activating Token Index: 990

Text #3

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.7636. Min Act: -0.0000

Data Index: 1022544 (The Pile)

Max Activating Token Index: 177

Text #4

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.7612. Min Act: -0.0000

Data Index: 1212714 (The Pile)

Max Activating Token Index: 172

Text #5

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.7725. Min Act: -0.0000

Data Index: 216074 (The Pile)

Max Activating Token Index: 281

Text #6

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.7157. Min Act: -0.0000

Data Index: 1052697 (The Pile)

Max Activating Token Index: 474

Text #7

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.6662. Min Act: -0.0000

Data Index: 1222984 (The Pile)

Max Activating Token Index: 978

Text #8

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.7236. Min Act: -0.0000

Data Index: 144058 (The Pile)

Max Activating Token Index: 478

Text #9

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.6737. Min Act: -0.0000

Data Index: 1308350 (The Pile)

Max Activating Token Index: 891

Text #10

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.6969. Min Act: -0.0000

Data Index: 281517 (The Pile)

Max Activating Token Index: 117

Text #11

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.6830. Min Act: -0.0000

Data Index: 1606517 (The Pile)

Max Activating Token Index: 715

Text #12

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.6791. Min Act: -0.0000

Data Index: 554487 (The Pile)

Max Activating Token Index: 418

Text #13

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.6613. Min Act: -0.0000

Data Index: 1761178 (The Pile)

Max Activating Token Index: 805

Text #14

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.6389. Min Act: -0.0000

Data Index: 1398314 (The Pile)

Max Activating Token Index: 744

Text #15

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.6621. Min Act: -0.0000

Data Index: 915755 (The Pile)

Max Activating Token Index: 337

Text #16

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.6145. Min Act: -0.0000

Data Index: 1565556 (The Pile)

Max Activating Token Index: 891

Text #17

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.6575. Min Act: -0.0000

Data Index: 1389420 (The Pile)

Max Activating Token Index: 617

Text #18

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.6003. Min Act: -0.0000

Data Index: 1166977 (The Pile)

Max Activating Token Index: 340

Text #19

Max Range: 0.9302. Min Range: -0.9302

Max Act: 0.5571. Min Act: -0.0000

Data Index: 323531 (The Pile)

Max Activating Token Index: 929