Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3581 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.9749. Min Range: -5.9749

Max Act: 5.9749. Min Act: -0.0000

Data Index: 967615 (The Pile)

Max Activating Token Index: 297

Text #1

Max Range: 5.9749. Min Range: -5.9749

Max Act: 5.4030. Min Act: -0.0000

Data Index: 609994 (The Pile)

Max Activating Token Index: 726

Text #2

Max Range: 5.9749. Min Range: -5.9749

Max Act: 5.7865. Min Act: -0.0000

Data Index: 1554262 (The Pile)

Max Activating Token Index: 766

Text #3

Max Range: 5.9749. Min Range: -5.9749

Max Act: 5.4146. Min Act: -0.0000

Data Index: 1404666 (The Pile)

Max Activating Token Index: 856

Text #4

Max Range: 5.9749. Min Range: -5.9749

Max Act: 5.0994. Min Act: -0.0000

Data Index: 1507925 (The Pile)

Max Activating Token Index: 250

Text #5

Max Range: 5.9749. Min Range: -5.9749

Max Act: 5.0806. Min Act: -0.0000

Data Index: 279955 (The Pile)

Max Activating Token Index: 171

Text #6

Max Range: 5.9749. Min Range: -5.9749

Max Act: 4.8735. Min Act: -0.0000

Data Index: 502206 (The Pile)

Max Activating Token Index: 476

Text #7

Max Range: 5.9749. Min Range: -5.9749

Max Act: 5.0352. Min Act: -0.0000

Data Index: 126482 (The Pile)

Max Activating Token Index: 914

Text #8

Max Range: 5.9749. Min Range: -5.9749

Max Act: 4.8380. Min Act: -0.0000

Data Index: 1455505 (The Pile)

Max Activating Token Index: 803

Text #9

Max Range: 5.9749. Min Range: -5.9749

Max Act: 5.0589. Min Act: -0.0000

Data Index: 1640206 (The Pile)

Max Activating Token Index: 448

Text #10

Max Range: 5.9749. Min Range: -5.9749

Max Act: 5.0321. Min Act: -0.0000

Data Index: 1554606 (The Pile)

Max Activating Token Index: 626

Text #11

Max Range: 5.9749. Min Range: -5.9749

Max Act: 4.8604. Min Act: -0.0000

Data Index: 363394 (The Pile)

Max Activating Token Index: 745

Text #12

Max Range: 5.9749. Min Range: -5.9749

Max Act: 4.8307. Min Act: -0.0000

Data Index: 371037 (The Pile)

Max Activating Token Index: 667

Text #13

Max Range: 5.9749. Min Range: -5.9749

Max Act: 5.0267. Min Act: -0.0000

Data Index: 831008 (The Pile)

Max Activating Token Index: 484

Text #14

Max Range: 5.9749. Min Range: -5.9749

Max Act: 5.1555. Min Act: -0.0000

Data Index: 147343 (The Pile)

Max Activating Token Index: 728

Text #15

Max Range: 5.9749. Min Range: -5.9749

Max Act: 4.6117. Min Act: -0.0000

Data Index: 856235 (The Pile)

Max Activating Token Index: 248

Text #16

Max Range: 5.9749. Min Range: -5.9749

Max Act: 4.7076. Min Act: -0.0000

Data Index: 1676597 (The Pile)

Max Activating Token Index: 274

Text #17

Max Range: 5.9749. Min Range: -5.9749

Max Act: 4.8539. Min Act: -0.0000

Data Index: 799844 (The Pile)

Max Activating Token Index: 1010

Text #18

Max Range: 5.9749. Min Range: -5.9749

Max Act: 4.5023. Min Act: -0.0000

Data Index: 543005 (The Pile)

Max Activating Token Index: 931

Text #19

Max Range: 5.9749. Min Range: -5.9749

Max Act: 4.4746. Min Act: -0.0000

Data Index: 1756638 (The Pile)

Max Activating Token Index: 78