Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3431 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0542. Min Range: -1.0542

Max Act: 1.0520. Min Act: -0.0000

Data Index: 473049 (The Pile)

Max Activating Token Index: 133

Text #1

Max Range: 1.0542. Min Range: -1.0542

Max Act: 1.0542. Min Act: -0.0000

Data Index: 463105 (The Pile)

Max Activating Token Index: 848

Text #2

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.8552. Min Act: -0.0000

Data Index: 1047900 (The Pile)

Max Activating Token Index: 634

Text #3

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.9501. Min Act: -0.0000

Data Index: 794544 (The Pile)

Max Activating Token Index: 66

Text #4

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.9082. Min Act: -0.0000

Data Index: 729489 (The Pile)

Max Activating Token Index: 175

Text #5

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.9445. Min Act: -0.0000

Data Index: 51776 (The Pile)

Max Activating Token Index: 86

Text #6

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.8939. Min Act: -0.0000

Data Index: 1047297 (The Pile)

Max Activating Token Index: 913

Text #7

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.8422. Min Act: -0.0000

Data Index: 1892824 (The Pile)

Max Activating Token Index: 94

Text #8

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.8632. Min Act: -0.0000

Data Index: 1598519 (The Pile)

Max Activating Token Index: 461

Text #9

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.8109. Min Act: -0.0000

Data Index: 589211 (The Pile)

Max Activating Token Index: 743

Text #10

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.8845. Min Act: -0.0000

Data Index: 1213493 (The Pile)

Max Activating Token Index: 210

Text #11

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.7771. Min Act: -0.0000

Data Index: 1445732 (The Pile)

Max Activating Token Index: 322

Text #12

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.7544. Min Act: -0.0000

Data Index: 679812 (The Pile)

Max Activating Token Index: 799

Text #13

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.7215. Min Act: -0.0000

Data Index: 1077552 (The Pile)

Max Activating Token Index: 396

Text #14

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.7431. Min Act: -0.0000

Data Index: 856223 (The Pile)

Max Activating Token Index: 533

Text #15

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.7297. Min Act: -0.0000

Data Index: 1299278 (The Pile)

Max Activating Token Index: 426

Text #16

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.7345. Min Act: -0.0000

Data Index: 1046628 (The Pile)

Max Activating Token Index: 797

Text #17

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.7213. Min Act: -0.0000

Data Index: 467329 (The Pile)

Max Activating Token Index: 355

Text #18

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.7290. Min Act: -0.0000

Data Index: 590411 (The Pile)

Max Activating Token Index: 217

Text #19

Max Range: 1.0542. Min Range: -1.0542

Max Act: 0.6951. Min Act: -0.0000

Data Index: 486028 (The Pile)

Max Activating Token Index: 510