Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4974 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.9858. Min Act: -0.0000

Data Index: 508377 (The Pile)

Max Activating Token Index: 553

Text #1

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.7585. Min Act: -0.0000

Data Index: 46786 (The Pile)

Max Activating Token Index: 434

Text #2

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.5744. Min Act: -0.0000

Data Index: 1728999 (The Pile)

Max Activating Token Index: 764

Text #3

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.6709. Min Act: -0.0000

Data Index: 968303 (The Pile)

Max Activating Token Index: 489

Text #4

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.5706. Min Act: -0.0000

Data Index: 994639 (The Pile)

Max Activating Token Index: 686

Text #5

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.5558. Min Act: -0.0000

Data Index: 848818 (The Pile)

Max Activating Token Index: 25

Text #6

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.5970. Min Act: -0.0000

Data Index: 85313 (The Pile)

Max Activating Token Index: 884

Text #7

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.5224. Min Act: -0.0000

Data Index: 952745 (The Pile)

Max Activating Token Index: 813

Text #8

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.6023. Min Act: -0.0000

Data Index: 1542173 (The Pile)

Max Activating Token Index: 980

Text #9

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.5872. Min Act: -0.0000

Data Index: 638481 (The Pile)

Max Activating Token Index: 709

Text #10

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.5061. Min Act: -0.0000

Data Index: 1891417 (The Pile)

Max Activating Token Index: 816

Text #11

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.5173. Min Act: -0.0000

Data Index: 362678 (The Pile)

Max Activating Token Index: 423

Text #12

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.5839. Min Act: -0.0000

Data Index: 901658 (The Pile)

Max Activating Token Index: 764

Text #13

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.5715. Min Act: -0.0000

Data Index: 511264 (The Pile)

Max Activating Token Index: 867

Text #14

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.6066. Min Act: -0.0000

Data Index: 1584472 (The Pile)

Max Activating Token Index: 387

Text #15

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.5205. Min Act: -0.0000

Data Index: 944038 (The Pile)

Max Activating Token Index: 930

Text #16

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.5024. Min Act: -0.0000

Data Index: 1133508 (The Pile)

Max Activating Token Index: 719

Text #17

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.4938. Min Act: -0.0000

Data Index: 1835242 (The Pile)

Max Activating Token Index: 726

Text #18

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.4823. Min Act: -0.0000

Data Index: 1840500 (The Pile)

Max Activating Token Index: 694

Text #19

Max Range: 1.9858. Min Range: -1.9858

Max Act: 1.6917. Min Act: -0.0000

Data Index: 826886 (The Pile)

Max Activating Token Index: 726