Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4150 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.6568. Min Act: -0.0000

Data Index: 1497846 (The Pile)

Max Activating Token Index: 41

Text #1

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.6268. Min Act: -0.0000

Data Index: 599381 (The Pile)

Max Activating Token Index: 45

Text #2

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.6255. Min Act: -0.0000

Data Index: 1100248 (The Pile)

Max Activating Token Index: 426

Text #3

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.6161. Min Act: -0.0000

Data Index: 568527 (The Pile)

Max Activating Token Index: 278

Text #4

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.6293. Min Act: -0.0000

Data Index: 1904453 (The Pile)

Max Activating Token Index: 177

Text #5

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5891. Min Act: -0.0000

Data Index: 1283 (The Pile)

Max Activating Token Index: 442

Text #6

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5584. Min Act: -0.0000

Data Index: 1367027 (The Pile)

Max Activating Token Index: 189

Text #7

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5864. Min Act: -0.0000

Data Index: 1692967 (The Pile)

Max Activating Token Index: 424

Text #8

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5682. Min Act: -0.0000

Data Index: 1728298 (The Pile)

Max Activating Token Index: 473

Text #9

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5958. Min Act: -0.0000

Data Index: 148405 (The Pile)

Max Activating Token Index: 24

Text #10

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5794. Min Act: -0.0000

Data Index: 487741 (The Pile)

Max Activating Token Index: 434

Text #11

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5575. Min Act: -0.0000

Data Index: 444694 (The Pile)

Max Activating Token Index: 158

Text #12

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5535. Min Act: -0.0000

Data Index: 1449895 (The Pile)

Max Activating Token Index: 107

Text #13

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5550. Min Act: -0.0000

Data Index: 1860743 (The Pile)

Max Activating Token Index: 39

Text #14

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5276. Min Act: -0.0000

Data Index: 1325127 (The Pile)

Max Activating Token Index: 372

Text #15

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5208. Min Act: -0.0000

Data Index: 217275 (The Pile)

Max Activating Token Index: 152

Text #16

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5317. Min Act: -0.0000

Data Index: 1832620 (The Pile)

Max Activating Token Index: 78

Text #17

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5301. Min Act: -0.0000

Data Index: 1898725 (The Pile)

Max Activating Token Index: 273

Text #18

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.5281. Min Act: -0.0000

Data Index: 731738 (The Pile)

Max Activating Token Index: 714

Text #19

Max Range: 0.6568. Min Range: -0.6568

Max Act: 0.4931. Min Act: -0.0000

Data Index: 1081871 (The Pile)

Max Activating Token Index: 269