Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1853 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.8854. Min Range: -5.8854

Max Act: 5.8854. Min Act: -0.0000

Data Index: 497986 (The Pile)

Max Activating Token Index: 1022

Text #1

Max Range: 5.8854. Min Range: -5.8854

Max Act: 5.6183. Min Act: -0.0000

Data Index: 748391 (The Pile)

Max Activating Token Index: 494

Text #2

Max Range: 5.8854. Min Range: -5.8854

Max Act: 5.2034. Min Act: -0.0000

Data Index: 132692 (The Pile)

Max Activating Token Index: 102

Text #3

Max Range: 5.8854. Min Range: -5.8854

Max Act: 3.2311. Min Act: -0.0000

Data Index: 860883 (The Pile)

Max Activating Token Index: 346

Text #4

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.8080. Min Act: -0.0000

Data Index: 394173 (The Pile)

Max Activating Token Index: 863

Text #5

Max Range: 5.8854. Min Range: -5.8854

Max Act: 3.0317. Min Act: -0.0000

Data Index: 755614 (The Pile)

Max Activating Token Index: 158

Text #6

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.9671. Min Act: -0.0000

Data Index: 1097215 (The Pile)

Max Activating Token Index: 318

Text #7

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.5534. Min Act: -0.0000

Data Index: 1795303 (The Pile)

Max Activating Token Index: 635

Text #8

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.7635. Min Act: -0.0000

Data Index: 1710307 (The Pile)

Max Activating Token Index: 327

Text #9

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.6354. Min Act: -0.0000

Data Index: 1414175 (The Pile)

Max Activating Token Index: 279

Text #10

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.4636. Min Act: -0.0000

Data Index: 317389 (The Pile)

Max Activating Token Index: 595

Text #11

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.5412. Min Act: -0.0000

Data Index: 1497535 (The Pile)

Max Activating Token Index: 408

Text #12

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.5422. Min Act: -0.0000

Data Index: 1617010 (The Pile)

Max Activating Token Index: 677

Text #13

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.3886. Min Act: -0.0000

Data Index: 756394 (The Pile)

Max Activating Token Index: 129

Text #14

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.5040. Min Act: -0.0000

Data Index: 1056807 (The Pile)

Max Activating Token Index: 962

Text #15

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.3916. Min Act: -0.0000

Data Index: 1421220 (The Pile)

Max Activating Token Index: 990

Text #16

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.3989. Min Act: -0.0000

Data Index: 1320766 (The Pile)

Max Activating Token Index: 112

Text #17

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.5375. Min Act: -0.0000

Data Index: 1138426 (The Pile)

Max Activating Token Index: 45

Text #18

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.3986. Min Act: -0.0000

Data Index: 1571418 (The Pile)

Max Activating Token Index: 220

Text #19

Max Range: 5.8854. Min Range: -5.8854

Max Act: 2.4978. Min Act: -0.0000

Data Index: 1094645 (The Pile)

Max Activating Token Index: 965