Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2758 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.3720. Min Range: -3.3720

Max Act: 3.3720. Min Act: -0.0000

Data Index: 250147 (The Pile)

Max Activating Token Index: 393

Text #1

Max Range: 3.3720. Min Range: -3.3720

Max Act: 3.3462. Min Act: -0.0000

Data Index: 1477347 (The Pile)

Max Activating Token Index: 340

Text #2

Max Range: 3.3720. Min Range: -3.3720

Max Act: 3.1135. Min Act: -0.0000

Data Index: 822451 (The Pile)

Max Activating Token Index: 610

Text #3

Max Range: 3.3720. Min Range: -3.3720

Max Act: 3.1640. Min Act: -0.0000

Data Index: 1408033 (The Pile)

Max Activating Token Index: 572

Text #4

Max Range: 3.3720. Min Range: -3.3720

Max Act: 3.2732. Min Act: -0.0000

Data Index: 319476 (The Pile)

Max Activating Token Index: 533

Text #5

Max Range: 3.3720. Min Range: -3.3720

Max Act: 3.0887. Min Act: -0.0000

Data Index: 1236472 (The Pile)

Max Activating Token Index: 511

Text #6

Max Range: 3.3720. Min Range: -3.3720

Max Act: 3.0647. Min Act: -0.0000

Data Index: 578373 (The Pile)

Max Activating Token Index: 567

Text #7

Max Range: 3.3720. Min Range: -3.3720

Max Act: 3.0587. Min Act: -0.0000

Data Index: 124024 (The Pile)

Max Activating Token Index: 600

Text #8

Max Range: 3.3720. Min Range: -3.3720

Max Act: 2.9212. Min Act: -0.0000

Data Index: 1482229 (The Pile)

Max Activating Token Index: 609

Text #9

Max Range: 3.3720. Min Range: -3.3720

Max Act: 2.8327. Min Act: -0.0000

Data Index: 867967 (The Pile)

Max Activating Token Index: 117

Text #10

Max Range: 3.3720. Min Range: -3.3720

Max Act: 2.7974. Min Act: -0.0000

Data Index: 669775 (The Pile)

Max Activating Token Index: 476

Text #11

Max Range: 3.3720. Min Range: -3.3720

Max Act: 3.1040. Min Act: -0.0000

Data Index: 1671304 (The Pile)

Max Activating Token Index: 447

Text #12

Max Range: 3.3720. Min Range: -3.3720

Max Act: 2.7945. Min Act: -0.0000

Data Index: 707710 (The Pile)

Max Activating Token Index: 747

Text #13

Max Range: 3.3720. Min Range: -3.3720

Max Act: 3.2266. Min Act: -0.0000

Data Index: 253191 (The Pile)

Max Activating Token Index: 541

Text #14

Max Range: 3.3720. Min Range: -3.3720

Max Act: 2.7942. Min Act: -0.0000

Data Index: 178740 (The Pile)

Max Activating Token Index: 6

Text #15

Max Range: 3.3720. Min Range: -3.3720

Max Act: 3.0704. Min Act: -0.0000

Data Index: 584565 (The Pile)

Max Activating Token Index: 531

Text #16

Max Range: 3.3720. Min Range: -3.3720

Max Act: 3.0542. Min Act: -0.0000

Data Index: 534849 (The Pile)

Max Activating Token Index: 117

Text #17

Max Range: 3.3720. Min Range: -3.3720

Max Act: 2.9310. Min Act: -0.0000

Data Index: 1709437 (The Pile)

Max Activating Token Index: 813

Text #18

Max Range: 3.3720. Min Range: -3.3720

Max Act: 2.8791. Min Act: -0.0000

Data Index: 1433456 (The Pile)

Max Activating Token Index: 145

Text #19

Max Range: 3.3720. Min Range: -3.3720

Max Act: 2.8854. Min Act: -0.0000

Data Index: 1940754 (The Pile)

Max Activating Token Index: 646