Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3076 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6114. Min Range: -1.6114

Max Act: 1.6114. Min Act: -0.0000

Data Index: 1545723 (The Pile)

Max Activating Token Index: 826

Text #1

Max Range: 1.6114. Min Range: -1.6114

Max Act: 1.1893. Min Act: 0.0001

Data Index: 1090698 (The Pile)

Max Activating Token Index: 773

Text #2

Max Range: 1.6114. Min Range: -1.6114

Max Act: 1.1369. Min Act: -0.0000

Data Index: 571408 (The Pile)

Max Activating Token Index: 709

Text #3

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.9976. Min Act: -0.0000

Data Index: 1032811 (The Pile)

Max Activating Token Index: 571

Text #4

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.9914. Min Act: -0.0000

Data Index: 1405488 (The Pile)

Max Activating Token Index: 876

Text #5

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.9580. Min Act: -0.0000

Data Index: 848449 (The Pile)

Max Activating Token Index: 930

Text #6

Max Range: 1.6114. Min Range: -1.6114

Max Act: 1.0033. Min Act: -0.0000

Data Index: 955437 (The Pile)

Max Activating Token Index: 557

Text #7

Max Range: 1.6114. Min Range: -1.6114

Max Act: 1.1382. Min Act: -0.0000

Data Index: 527461 (The Pile)

Max Activating Token Index: 1011

Text #8

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.9050. Min Act: -0.0000

Data Index: 918591 (The Pile)

Max Activating Token Index: 772

Text #9

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.9414. Min Act: -0.0000

Data Index: 1783552 (The Pile)

Max Activating Token Index: 287

Text #10

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.9461. Min Act: -0.0000

Data Index: 1860610 (The Pile)

Max Activating Token Index: 218

Text #11

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.8798. Min Act: -0.0000

Data Index: 613518 (The Pile)

Max Activating Token Index: 315

Text #12

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.8558. Min Act: 0.0000

Data Index: 1310318 (The Pile)

Max Activating Token Index: 1011

Text #13

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.8499. Min Act: -0.0000

Data Index: 1823763 (The Pile)

Max Activating Token Index: 741

Text #14

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.8495. Min Act: -0.0000

Data Index: 1547381 (The Pile)

Max Activating Token Index: 776

Text #15

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.9060. Min Act: -0.0000

Data Index: 467571 (The Pile)

Max Activating Token Index: 298

Text #16

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.8621. Min Act: -0.0000

Data Index: 602380 (The Pile)

Max Activating Token Index: 241

Text #17

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.7953. Min Act: -0.0000

Data Index: 1470968 (The Pile)

Max Activating Token Index: 678

Text #18

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.9205. Min Act: -0.0000

Data Index: 794726 (The Pile)

Max Activating Token Index: 227

Text #19

Max Range: 1.6114. Min Range: -1.6114

Max Act: 0.7537. Min Act: -0.0000

Data Index: 792684 (The Pile)

Max Activating Token Index: 757