Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3123 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.5124. Min Act: -0.0000

Data Index: 983619 (The Pile)

Max Activating Token Index: 294

Text #1

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.4910. Min Act: -0.0000

Data Index: 1805381 (The Pile)

Max Activating Token Index: 662

Text #2

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.4787. Min Act: -0.0000

Data Index: 109822 (The Pile)

Max Activating Token Index: 69

Text #3

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.4834. Min Act: -0.0000

Data Index: 284961 (The Pile)

Max Activating Token Index: 169

Text #4

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.4436. Min Act: -0.0000

Data Index: 1536666 (The Pile)

Max Activating Token Index: 855

Text #5

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.4112. Min Act: -0.0000

Data Index: 409583 (The Pile)

Max Activating Token Index: 955

Text #6

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.3960. Min Act: -0.0000

Data Index: 1995462 (The Pile)

Max Activating Token Index: 500

Text #7

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.3859. Min Act: -0.0000

Data Index: 484389 (The Pile)

Max Activating Token Index: 844

Text #8

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.4003. Min Act: -0.0000

Data Index: 32092 (The Pile)

Max Activating Token Index: 604

Text #9

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.4050. Min Act: -0.0000

Data Index: 579552 (The Pile)

Max Activating Token Index: 702

Text #10

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.4243. Min Act: -0.0000

Data Index: 1247551 (The Pile)

Max Activating Token Index: 1004

Text #11

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.4267. Min Act: -0.0000

Data Index: 151832 (The Pile)

Max Activating Token Index: 511

Text #12

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.4166. Min Act: -0.0000

Data Index: 1457990 (The Pile)

Max Activating Token Index: 347

Text #13

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.3928. Min Act: -0.0000

Data Index: 657563 (The Pile)

Max Activating Token Index: 970

Text #14

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.4219. Min Act: -0.0000

Data Index: 1839161 (The Pile)

Max Activating Token Index: 488

Text #15

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.3727. Min Act: -0.0000

Data Index: 1290689 (The Pile)

Max Activating Token Index: 943

Text #16

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.3700. Min Act: -0.0000

Data Index: 1614050 (The Pile)

Max Activating Token Index: 906

Text #17

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.3731. Min Act: -0.0000

Data Index: 1349390 (The Pile)

Max Activating Token Index: 828

Text #18

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.3653. Min Act: -0.0000

Data Index: 1079529 (The Pile)

Max Activating Token Index: 86

Text #19

Max Range: 0.5124. Min Range: -0.5124

Max Act: 0.3806. Min Act: -0.0000

Data Index: 1681539 (The Pile)

Max Activating Token Index: 250