Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2062 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.5910. Min Range: -3.5910

Max Act: 3.3627. Min Act: -0.0000

Data Index: 51040 (The Pile)

Max Activating Token Index: 933

Text #1

Max Range: 3.5910. Min Range: -3.5910

Max Act: 3.5910. Min Act: -0.0000

Data Index: 500188 (The Pile)

Max Activating Token Index: 790

Text #2

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.7799. Min Act: -0.0000

Data Index: 1204298 (The Pile)

Max Activating Token Index: 869

Text #3

Max Range: 3.5910. Min Range: -3.5910

Max Act: 3.2419. Min Act: -0.0000

Data Index: 1599475 (The Pile)

Max Activating Token Index: 750

Text #4

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.7265. Min Act: -0.0000

Data Index: 1736120 (The Pile)

Max Activating Token Index: 974

Text #5

Max Range: 3.5910. Min Range: -3.5910

Max Act: 3.0640. Min Act: -0.0000

Data Index: 900742 (The Pile)

Max Activating Token Index: 493

Text #6

Max Range: 3.5910. Min Range: -3.5910

Max Act: 3.0004. Min Act: -0.0000

Data Index: 582240 (The Pile)

Max Activating Token Index: 46

Text #7

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.7140. Min Act: -0.0000

Data Index: 80483 (The Pile)

Max Activating Token Index: 749

Text #8

Max Range: 3.5910. Min Range: -3.5910

Max Act: 3.0994. Min Act: -0.0000

Data Index: 742393 (The Pile)

Max Activating Token Index: 78

Text #9

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.8513. Min Act: -0.0000

Data Index: 1641436 (The Pile)

Max Activating Token Index: 833

Text #10

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.9383. Min Act: -0.0000

Data Index: 1932022 (The Pile)

Max Activating Token Index: 702

Text #11

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.7542. Min Act: -0.0000

Data Index: 548989 (The Pile)

Max Activating Token Index: 287

Text #12

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.8177. Min Act: -0.0000

Data Index: 1236743 (The Pile)

Max Activating Token Index: 1007

Text #13

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.6184. Min Act: -0.0000

Data Index: 235673 (The Pile)

Max Activating Token Index: 969

Text #14

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.3704. Min Act: -0.0000

Data Index: 861247 (The Pile)

Max Activating Token Index: 713

Text #15

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.4915. Min Act: -0.0000

Data Index: 206961 (The Pile)

Max Activating Token Index: 110

Text #16

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.6830. Min Act: -0.0000

Data Index: 330849 (The Pile)

Max Activating Token Index: 885

Text #17

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.5722. Min Act: -0.0000

Data Index: 1503830 (The Pile)

Max Activating Token Index: 489

Text #18

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.5678. Min Act: -0.0000

Data Index: 889737 (The Pile)

Max Activating Token Index: 34

Text #19

Max Range: 3.5910. Min Range: -3.5910

Max Act: 2.6063. Min Act: -0.0000

Data Index: 1386636 (The Pile)

Max Activating Token Index: 436