Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2373 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.4756. Min Act: -0.0000

Data Index: 1320750 (The Pile)

Max Activating Token Index: 349

Text #1

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.4031. Min Act: -0.0000

Data Index: 435121 (The Pile)

Max Activating Token Index: 434

Text #2

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3767. Min Act: -0.0000

Data Index: 1106800 (The Pile)

Max Activating Token Index: 774

Text #3

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3547. Min Act: -0.0000

Data Index: 1343596 (The Pile)

Max Activating Token Index: 921

Text #4

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3738. Min Act: -0.0000

Data Index: 88516 (The Pile)

Max Activating Token Index: 100

Text #5

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3674. Min Act: -0.0000

Data Index: 337146 (The Pile)

Max Activating Token Index: 906

Text #6

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3318. Min Act: -0.0000

Data Index: 364999 (The Pile)

Max Activating Token Index: 852

Text #7

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.2902. Min Act: -0.0000

Data Index: 988107 (The Pile)

Max Activating Token Index: 1015

Text #8

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3533. Min Act: -0.0000

Data Index: 913754 (The Pile)

Max Activating Token Index: 872

Text #9

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3212. Min Act: -0.0000

Data Index: 838180 (The Pile)

Max Activating Token Index: 119

Text #10

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.2959. Min Act: -0.0000

Data Index: 1392732 (The Pile)

Max Activating Token Index: 439

Text #11

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3516. Min Act: -0.0000

Data Index: 1215580 (The Pile)

Max Activating Token Index: 919

Text #12

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3482. Min Act: -0.0000

Data Index: 102416 (The Pile)

Max Activating Token Index: 841

Text #13

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3380. Min Act: -0.0000

Data Index: 595999 (The Pile)

Max Activating Token Index: 811

Text #14

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.2968. Min Act: -0.0000

Data Index: 515480 (The Pile)

Max Activating Token Index: 302

Text #15

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3285. Min Act: -0.0000

Data Index: 1941584 (The Pile)

Max Activating Token Index: 238

Text #16

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3029. Min Act: -0.0000

Data Index: 946109 (The Pile)

Max Activating Token Index: 475

Text #17

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3072. Min Act: -0.0000

Data Index: 1178103 (The Pile)

Max Activating Token Index: 523

Text #18

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3117. Min Act: -0.0000

Data Index: 510677 (The Pile)

Max Activating Token Index: 834

Text #19

Max Range: 0.4756. Min Range: -0.4756

Max Act: 0.3069. Min Act: -0.0000

Data Index: 809314 (The Pile)

Max Activating Token Index: 550