Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3761 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.8218. Min Act: -0.0000

Data Index: 974961 (The Pile)

Max Activating Token Index: 1009

Text #1

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.7926. Min Act: -0.0000

Data Index: 1364912 (The Pile)

Max Activating Token Index: 463

Text #2

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.7890. Min Act: -0.0000

Data Index: 498057 (The Pile)

Max Activating Token Index: 712

Text #3

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.7785. Min Act: -0.0000

Data Index: 1903183 (The Pile)

Max Activating Token Index: 377

Text #4

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.7388. Min Act: -0.0000

Data Index: 1227406 (The Pile)

Max Activating Token Index: 531

Text #5

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.7279. Min Act: -0.0000

Data Index: 499796 (The Pile)

Max Activating Token Index: 914

Text #6

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.6841. Min Act: -0.0000

Data Index: 1517806 (The Pile)

Max Activating Token Index: 591

Text #7

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.7046. Min Act: 0.0000

Data Index: 1399626 (The Pile)

Max Activating Token Index: 771

Text #8

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.7335. Min Act: -0.0000

Data Index: 824366 (The Pile)

Max Activating Token Index: 878

Text #9

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.6641. Min Act: -0.0000

Data Index: 2535 (The Pile)

Max Activating Token Index: 879

Text #10

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.6907. Min Act: -0.0000

Data Index: 415844 (The Pile)

Max Activating Token Index: 364

Text #11

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.6745. Min Act: -0.0000

Data Index: 193438 (The Pile)

Max Activating Token Index: 813

Text #12

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.6691. Min Act: -0.0000

Data Index: 726906 (The Pile)

Max Activating Token Index: 334

Text #13

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.6650. Min Act: -0.0000

Data Index: 740599 (The Pile)

Max Activating Token Index: 89

Text #14

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.6646. Min Act: -0.0000

Data Index: 1441587 (The Pile)

Max Activating Token Index: 447

Text #15

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.6859. Min Act: -0.0000

Data Index: 1877059 (The Pile)

Max Activating Token Index: 124

Text #16

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.6473. Min Act: -0.0000

Data Index: 1648303 (The Pile)

Max Activating Token Index: 974

Text #17

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.6024. Min Act: -0.0000

Data Index: 4899 (The Pile)

Max Activating Token Index: 730

Text #18

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.6261. Min Act: -0.0000

Data Index: 832829 (The Pile)

Max Activating Token Index: 1016

Text #19

Max Range: 0.8218. Min Range: -0.8218

Max Act: 0.6215. Min Act: -0.0000

Data Index: 1641313 (The Pile)

Max Activating Token Index: 820