Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 586 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.6801. Min Act: -0.0000

Data Index: 1411937 (The Pile)

Max Activating Token Index: 824

Text #1

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.7018. Min Act: -0.0000

Data Index: 1169636 (The Pile)

Max Activating Token Index: 113

Text #2

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.6450. Min Act: -0.0000

Data Index: 215095 (The Pile)

Max Activating Token Index: 33

Text #3

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.6288. Min Act: -0.0000

Data Index: 314702 (The Pile)

Max Activating Token Index: 792

Text #4

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.5750. Min Act: -0.0000

Data Index: 1888394 (The Pile)

Max Activating Token Index: 669

Text #5

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.5938. Min Act: -0.0000

Data Index: 1308996 (The Pile)

Max Activating Token Index: 429

Text #6

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.5184. Min Act: -0.0000

Data Index: 1151785 (The Pile)

Max Activating Token Index: 698

Text #7

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.4988. Min Act: -0.0000

Data Index: 1374752 (The Pile)

Max Activating Token Index: 701

Text #8

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.4848. Min Act: -0.0000

Data Index: 1376710 (The Pile)

Max Activating Token Index: 88

Text #9

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.4834. Min Act: -0.0000

Data Index: 1054845 (The Pile)

Max Activating Token Index: 261

Text #10

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.5161. Min Act: -0.0000

Data Index: 1932510 (The Pile)

Max Activating Token Index: 626

Text #11

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.4876. Min Act: -0.0000

Data Index: 594602 (The Pile)

Max Activating Token Index: 938

Text #12

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.4695. Min Act: -0.0000

Data Index: 187033 (The Pile)

Max Activating Token Index: 840

Text #13

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.4960. Min Act: -0.0000

Data Index: 636743 (The Pile)

Max Activating Token Index: 751

Text #14

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.4840. Min Act: -0.0000

Data Index: 1680742 (The Pile)

Max Activating Token Index: 362

Text #15

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.4960. Min Act: -0.0000

Data Index: 243547 (The Pile)

Max Activating Token Index: 231

Text #16

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.4952. Min Act: -0.0000

Data Index: 1754818 (The Pile)

Max Activating Token Index: 568

Text #17

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.4766. Min Act: -0.0000

Data Index: 1991204 (The Pile)

Max Activating Token Index: 941

Text #18

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.4897. Min Act: -0.0000

Data Index: 1846710 (The Pile)

Max Activating Token Index: 23

Text #19

Max Range: 0.7018. Min Range: -0.7018

Max Act: 0.4855. Min Act: -0.0000

Data Index: 1835358 (The Pile)

Max Activating Token Index: 597