Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 173 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.5138. Min Act: -0.0000

Data Index: 487636 (The Pile)

Max Activating Token Index: 300

Text #1

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.3789. Min Act: -0.0000

Data Index: 1714239 (The Pile)

Max Activating Token Index: 184

Text #2

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.3778. Min Act: -0.0000

Data Index: 712418 (The Pile)

Max Activating Token Index: 720

Text #3

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.4503. Min Act: -0.0000

Data Index: 632212 (The Pile)

Max Activating Token Index: 47

Text #4

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.4525. Min Act: -0.0000

Data Index: 696844 (The Pile)

Max Activating Token Index: 114

Text #5

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.5432. Min Act: -0.0000

Data Index: 305009 (The Pile)

Max Activating Token Index: 1002

Text #6

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.3987. Min Act: -0.0000

Data Index: 1374525 (The Pile)

Max Activating Token Index: 135

Text #7

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.3841. Min Act: -0.0000

Data Index: 622115 (The Pile)

Max Activating Token Index: 123

Text #8

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.3814. Min Act: -0.0000

Data Index: 1485211 (The Pile)

Max Activating Token Index: 184

Text #9

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.3771. Min Act: -0.0000

Data Index: 334902 (The Pile)

Max Activating Token Index: 455

Text #10

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.3654. Min Act: -0.0000

Data Index: 1032721 (The Pile)

Max Activating Token Index: 399

Text #11

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.3680. Min Act: -0.0000

Data Index: 508465 (The Pile)

Max Activating Token Index: 530

Text #12

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.3623. Min Act: -0.0000

Data Index: 843700 (The Pile)

Max Activating Token Index: 461

Text #13

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.2290. Min Act: -0.0000

Data Index: 900969 (The Pile)

Max Activating Token Index: 297

Text #14

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.3605. Min Act: -0.0000

Data Index: 739709 (The Pile)

Max Activating Token Index: 747

Text #15

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.2182. Min Act: -0.0000

Data Index: 505283 (The Pile)

Max Activating Token Index: 605

Text #16

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.1767. Min Act: -0.0000

Data Index: 1636546 (The Pile)

Max Activating Token Index: 130

Text #17

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.2843. Min Act: -0.0000

Data Index: 474862 (The Pile)

Max Activating Token Index: 398

Text #18

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.2653. Min Act: -0.0000

Data Index: 1325605 (The Pile)

Max Activating Token Index: 390

Text #19

Max Range: 2.5432. Min Range: -2.5432

Max Act: 2.3884. Min Act: -0.0000

Data Index: 1296414 (The Pile)

Max Activating Token Index: 101