Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3722 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2911. Min Act: -0.0000

Data Index: 1338795 (The Pile)

Max Activating Token Index: 1003

Text #1

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2671. Min Act: -0.0000

Data Index: 1510865 (The Pile)

Max Activating Token Index: 849

Text #2

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2841. Min Act: -0.0000

Data Index: 94450 (The Pile)

Max Activating Token Index: 985

Text #3

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2723. Min Act: -0.0000

Data Index: 1879439 (The Pile)

Max Activating Token Index: 556

Text #4

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2641. Min Act: -0.0000

Data Index: 1048030 (The Pile)

Max Activating Token Index: 440

Text #5

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2150. Min Act: -0.0000

Data Index: 86264 (The Pile)

Max Activating Token Index: 843

Text #6

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2494. Min Act: -0.0000

Data Index: 41468 (The Pile)

Max Activating Token Index: 889

Text #7

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2296. Min Act: -0.0000

Data Index: 750303 (The Pile)

Max Activating Token Index: 276

Text #8

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2278. Min Act: -0.0000

Data Index: 741103 (The Pile)

Max Activating Token Index: 276

Text #9

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2270. Min Act: -0.0000

Data Index: 1889678 (The Pile)

Max Activating Token Index: 340

Text #10

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2244. Min Act: -0.0000

Data Index: 1718671 (The Pile)

Max Activating Token Index: 50

Text #11

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2481. Min Act: -0.0000

Data Index: 485889 (The Pile)

Max Activating Token Index: 92

Text #12

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2146. Min Act: -0.0000

Data Index: 1205536 (The Pile)

Max Activating Token Index: 802

Text #13

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2308. Min Act: -0.0000

Data Index: 603461 (The Pile)

Max Activating Token Index: 349

Text #14

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.1995. Min Act: -0.0000

Data Index: 1149960 (The Pile)

Max Activating Token Index: 1006

Text #15

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2153. Min Act: -0.0000

Data Index: 1402999 (The Pile)

Max Activating Token Index: 956

Text #16

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2150. Min Act: -0.0000

Data Index: 62596 (The Pile)

Max Activating Token Index: 858

Text #17

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2062. Min Act: -0.0000

Data Index: 1271095 (The Pile)

Max Activating Token Index: 317

Text #18

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2124. Min Act: -0.0000

Data Index: 1498076 (The Pile)

Max Activating Token Index: 581

Text #19

Max Range: 0.2911. Min Range: -0.2911

Max Act: 0.2026. Min Act: -0.0000

Data Index: 1056590 (The Pile)

Max Activating Token Index: 325