Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2705 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.1616. Min Range: -3.1616

Max Act: 3.1270. Min Act: -0.0000

Data Index: 851952 (The Pile)

Max Activating Token Index: 314

Text #1

Max Range: 3.1616. Min Range: -3.1616

Max Act: 3.1616. Min Act: -0.0000

Data Index: 1177951 (The Pile)

Max Activating Token Index: 269

Text #2

Max Range: 3.1616. Min Range: -3.1616

Max Act: 3.0103. Min Act: -0.0000

Data Index: 1335115 (The Pile)

Max Activating Token Index: 78

Text #3

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.9582. Min Act: -0.0000

Data Index: 1630631 (The Pile)

Max Activating Token Index: 413

Text #4

Max Range: 3.1616. Min Range: -3.1616

Max Act: 3.0029. Min Act: -0.0000

Data Index: 697598 (The Pile)

Max Activating Token Index: 109

Text #5

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.8537. Min Act: -0.0000

Data Index: 942930 (The Pile)

Max Activating Token Index: 552

Text #6

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.9133. Min Act: -0.0000

Data Index: 107049 (The Pile)

Max Activating Token Index: 534

Text #7

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.6362. Min Act: -0.0000

Data Index: 136477 (The Pile)

Max Activating Token Index: 336

Text #8

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.2518. Min Act: -0.0000

Data Index: 1822985 (The Pile)

Max Activating Token Index: 642

Text #9

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.3060. Min Act: -0.0000

Data Index: 676349 (The Pile)

Max Activating Token Index: 910

Text #10

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.2006. Min Act: -0.0000

Data Index: 467211 (The Pile)

Max Activating Token Index: 706

Text #11

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.1852. Min Act: -0.0000

Data Index: 731522 (The Pile)

Max Activating Token Index: 717

Text #12

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.2564. Min Act: -0.0000

Data Index: 1057766 (The Pile)

Max Activating Token Index: 619

Text #13

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.3481. Min Act: -0.0000

Data Index: 1371388 (The Pile)

Max Activating Token Index: 569

Text #14

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.3360. Min Act: -0.0000

Data Index: 1922503 (The Pile)

Max Activating Token Index: 154

Text #15

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.3524. Min Act: -0.0000

Data Index: 166962 (The Pile)

Max Activating Token Index: 920

Text #16

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.3306. Min Act: -0.0000

Data Index: 1782162 (The Pile)

Max Activating Token Index: 775

Text #17

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.0862. Min Act: -0.0000

Data Index: 1090372 (The Pile)

Max Activating Token Index: 889

Text #18

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.2828. Min Act: -0.0000

Data Index: 1412677 (The Pile)

Max Activating Token Index: 907

Text #19

Max Range: 3.1616. Min Range: -3.1616

Max Act: 2.1470. Min Act: -0.0000

Data Index: 782441 (The Pile)

Max Activating Token Index: 599