Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3583 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.2727. Min Act: -0.0000

Data Index: 597085 (The Pile)

Max Activating Token Index: 738

Text #1

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.2213. Min Act: -0.0000

Data Index: 689276 (The Pile)

Max Activating Token Index: 357

Text #2

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1817. Min Act: -0.0000

Data Index: 993158 (The Pile)

Max Activating Token Index: 429

Text #3

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1838. Min Act: -0.0000

Data Index: 211482 (The Pile)

Max Activating Token Index: 79

Text #4

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1871. Min Act: -0.0000

Data Index: 441080 (The Pile)

Max Activating Token Index: 876

Text #5

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1872. Min Act: -0.0000

Data Index: 119195 (The Pile)

Max Activating Token Index: 721

Text #6

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1873. Min Act: -0.0000

Data Index: 396863 (The Pile)

Max Activating Token Index: 767

Text #7

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1772. Min Act: -0.0000

Data Index: 1465878 (The Pile)

Max Activating Token Index: 692

Text #8

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1743. Min Act: -0.0000

Data Index: 1362880 (The Pile)

Max Activating Token Index: 119

Text #9

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1870. Min Act: -0.0000

Data Index: 1217642 (The Pile)

Max Activating Token Index: 207

Text #10

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1796. Min Act: -0.0000

Data Index: 335173 (The Pile)

Max Activating Token Index: 491

Text #11

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1729. Min Act: -0.0000

Data Index: 1171702 (The Pile)

Max Activating Token Index: 663

Text #12

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1520. Min Act: -0.0000

Data Index: 1872934 (The Pile)

Max Activating Token Index: 767

Text #13

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1864. Min Act: -0.0000

Data Index: 1978463 (The Pile)

Max Activating Token Index: 835

Text #14

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1759. Min Act: -0.0000

Data Index: 726676 (The Pile)

Max Activating Token Index: 653

Text #15

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1750. Min Act: -0.0000

Data Index: 202186 (The Pile)

Max Activating Token Index: 403

Text #16

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1683. Min Act: -0.0000

Data Index: 1805091 (The Pile)

Max Activating Token Index: 959

Text #17

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1584. Min Act: -0.0000

Data Index: 536610 (The Pile)

Max Activating Token Index: 771

Text #18

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1784. Min Act: -0.0000

Data Index: 438530 (The Pile)

Max Activating Token Index: 285

Text #19

Max Range: 0.2727. Min Range: -0.2727

Max Act: 0.1602. Min Act: -0.0000

Data Index: 813777 (The Pile)

Max Activating Token Index: 923