Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1441 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.5572. Min Act: -0.0000

Data Index: 1976929 (The Pile)

Max Activating Token Index: 428

Text #1

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.5614. Min Act: -0.0000

Data Index: 256619 (The Pile)

Max Activating Token Index: 364

Text #2

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.5271. Min Act: -0.0000

Data Index: 1608546 (The Pile)

Max Activating Token Index: 39

Text #3

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4988. Min Act: -0.0000

Data Index: 904751 (The Pile)

Max Activating Token Index: 887

Text #4

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4786. Min Act: -0.0000

Data Index: 786202 (The Pile)

Max Activating Token Index: 1010

Text #5

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.5116. Min Act: -0.0000

Data Index: 1431503 (The Pile)

Max Activating Token Index: 34

Text #6

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.5301. Min Act: -0.0000

Data Index: 1358078 (The Pile)

Max Activating Token Index: 934

Text #7

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4964. Min Act: -0.0000

Data Index: 687084 (The Pile)

Max Activating Token Index: 878

Text #8

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.5078. Min Act: -0.0000

Data Index: 1647787 (The Pile)

Max Activating Token Index: 257

Text #9

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4944. Min Act: -0.0000

Data Index: 370080 (The Pile)

Max Activating Token Index: 310

Text #10

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.5045. Min Act: -0.0000

Data Index: 839391 (The Pile)

Max Activating Token Index: 834

Text #11

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4749. Min Act: -0.0000

Data Index: 490165 (The Pile)

Max Activating Token Index: 372

Text #12

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4861. Min Act: -0.0000

Data Index: 984095 (The Pile)

Max Activating Token Index: 679

Text #13

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4680. Min Act: -0.0000

Data Index: 1124881 (The Pile)

Max Activating Token Index: 560

Text #14

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4651. Min Act: -0.0000

Data Index: 1030591 (The Pile)

Max Activating Token Index: 129

Text #15

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4808. Min Act: -0.0000

Data Index: 322867 (The Pile)

Max Activating Token Index: 168

Text #16

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4583. Min Act: -0.0000

Data Index: 832123 (The Pile)

Max Activating Token Index: 277

Text #17

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4619. Min Act: -0.0000

Data Index: 1018283 (The Pile)

Max Activating Token Index: 706

Text #18

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4570. Min Act: -0.0000

Data Index: 1897265 (The Pile)

Max Activating Token Index: 644

Text #19

Max Range: 0.5614. Min Range: -0.5614

Max Act: 0.4618. Min Act: -0.0000

Data Index: 572763 (The Pile)

Max Activating Token Index: 880