Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2863 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.9156. Min Act: -0.0000

Data Index: 1620254 (The Pile)

Max Activating Token Index: 528

Text #1

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.9686. Min Act: -0.0000

Data Index: 1680172 (The Pile)

Max Activating Token Index: 365

Text #2

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7974. Min Act: -0.0000

Data Index: 1951267 (The Pile)

Max Activating Token Index: 1010

Text #3

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.8453. Min Act: -0.0000

Data Index: 1343307 (The Pile)

Max Activating Token Index: 110

Text #4

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.8583. Min Act: -0.0000

Data Index: 905098 (The Pile)

Max Activating Token Index: 412

Text #5

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.8248. Min Act: 0.0000

Data Index: 1737698 (The Pile)

Max Activating Token Index: 184

Text #6

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7957. Min Act: -0.0000

Data Index: 529627 (The Pile)

Max Activating Token Index: 562

Text #7

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.8193. Min Act: -0.0000

Data Index: 1008176 (The Pile)

Max Activating Token Index: 295

Text #8

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7689. Min Act: -0.0000

Data Index: 382974 (The Pile)

Max Activating Token Index: 791

Text #9

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7835. Min Act: 0.0001

Data Index: 583528 (The Pile)

Max Activating Token Index: 328

Text #10

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7911. Min Act: -0.0000

Data Index: 953510 (The Pile)

Max Activating Token Index: 669

Text #11

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7147. Min Act: 0.0000

Data Index: 1223601 (The Pile)

Max Activating Token Index: 872

Text #12

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7086. Min Act: -0.0000

Data Index: 1289475 (The Pile)

Max Activating Token Index: 897

Text #13

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7243. Min Act: -0.0000

Data Index: 1500773 (The Pile)

Max Activating Token Index: 749

Text #14

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7299. Min Act: -0.0000

Data Index: 1365021 (The Pile)

Max Activating Token Index: 89

Text #15

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7530. Min Act: -0.0000

Data Index: 1603882 (The Pile)

Max Activating Token Index: 461

Text #16

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7395. Min Act: -0.0000

Data Index: 263822 (The Pile)

Max Activating Token Index: 745

Text #17

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7467. Min Act: -0.0000

Data Index: 1181157 (The Pile)

Max Activating Token Index: 936

Text #18

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.7407. Min Act: -0.0000

Data Index: 58510 (The Pile)

Max Activating Token Index: 336

Text #19

Max Range: 0.9686. Min Range: -0.9686

Max Act: 0.6991. Min Act: -0.0000

Data Index: 1636064 (The Pile)

Max Activating Token Index: 412