Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1888 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.5857. Min Act: -0.0000

Data Index: 1266486 (The Pile)

Max Activating Token Index: 913

Text #1

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.5093. Min Act: -0.0000

Data Index: 203799 (The Pile)

Max Activating Token Index: 1012

Text #2

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.5474. Min Act: -0.0000

Data Index: 1196403 (The Pile)

Max Activating Token Index: 228

Text #3

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.4646. Min Act: -0.0000

Data Index: 1879639 (The Pile)

Max Activating Token Index: 925

Text #4

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.5587. Min Act: -0.0000

Data Index: 553900 (The Pile)

Max Activating Token Index: 73

Text #5

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.5035. Min Act: -0.0000

Data Index: 1713555 (The Pile)

Max Activating Token Index: 449

Text #6

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.4641. Min Act: -0.0000

Data Index: 190971 (The Pile)

Max Activating Token Index: 505

Text #7

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.4659. Min Act: -0.0000

Data Index: 909350 (The Pile)

Max Activating Token Index: 135

Text #8

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.4798. Min Act: -0.0000

Data Index: 1571778 (The Pile)

Max Activating Token Index: 401

Text #9

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.4567. Min Act: -0.0000

Data Index: 1523750 (The Pile)

Max Activating Token Index: 266

Text #10

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.4692. Min Act: -0.0000

Data Index: 901277 (The Pile)

Max Activating Token Index: 394

Text #11

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.4026. Min Act: -0.0000

Data Index: 273266 (The Pile)

Max Activating Token Index: 572

Text #12

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.3711. Min Act: -0.0000

Data Index: 796896 (The Pile)

Max Activating Token Index: 603

Text #13

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.4622. Min Act: -0.0000

Data Index: 253906 (The Pile)

Max Activating Token Index: 805

Text #14

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.5088. Min Act: -0.0000

Data Index: 1800077 (The Pile)

Max Activating Token Index: 582

Text #15

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.4905. Min Act: -0.0000

Data Index: 1384405 (The Pile)

Max Activating Token Index: 457

Text #16

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.4572. Min Act: -0.0000

Data Index: 519587 (The Pile)

Max Activating Token Index: 176

Text #17

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.3751. Min Act: -0.0000

Data Index: 900168 (The Pile)

Max Activating Token Index: 656

Text #18

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.3800. Min Act: -0.0000

Data Index: 724677 (The Pile)

Max Activating Token Index: 660

Text #19

Max Range: 0.5857. Min Range: -0.5857

Max Act: 0.4258. Min Act: -0.0000

Data Index: 7041 (The Pile)

Max Activating Token Index: 632