Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2636 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.9606. Min Act: -0.0000

Data Index: 1352204 (The Pile)

Max Activating Token Index: 574

Text #1

Max Range: 1.0080. Min Range: -1.0080

Max Act: 1.0080. Min Act: -0.0000

Data Index: 1218125 (The Pile)

Max Activating Token Index: 969

Text #2

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.9833. Min Act: -0.0000

Data Index: 210587 (The Pile)

Max Activating Token Index: 733

Text #3

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.7795. Min Act: -0.0000

Data Index: 1796634 (The Pile)

Max Activating Token Index: 926

Text #4

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.7741. Min Act: -0.0000

Data Index: 1486085 (The Pile)

Max Activating Token Index: 520

Text #5

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.8700. Min Act: -0.0000

Data Index: 359020 (The Pile)

Max Activating Token Index: 686

Text #6

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.8316. Min Act: -0.0000

Data Index: 723584 (The Pile)

Max Activating Token Index: 378

Text #7

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.6994. Min Act: -0.0000

Data Index: 1141752 (The Pile)

Max Activating Token Index: 794

Text #8

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.7618. Min Act: -0.0000

Data Index: 841375 (The Pile)

Max Activating Token Index: 203

Text #9

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.6896. Min Act: -0.0000

Data Index: 591876 (The Pile)

Max Activating Token Index: 686

Text #10

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.7189. Min Act: -0.0000

Data Index: 179337 (The Pile)

Max Activating Token Index: 261

Text #11

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.8084. Min Act: -0.0000

Data Index: 493467 (The Pile)

Max Activating Token Index: 653

Text #12

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.6823. Min Act: -0.0000

Data Index: 101963 (The Pile)

Max Activating Token Index: 899

Text #13

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.7176. Min Act: -0.0000

Data Index: 626995 (The Pile)

Max Activating Token Index: 63

Text #14

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.7936. Min Act: -0.0000

Data Index: 42039 (The Pile)

Max Activating Token Index: 695

Text #15

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.6607. Min Act: -0.0000

Data Index: 1179041 (The Pile)

Max Activating Token Index: 605

Text #16

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.7512. Min Act: -0.0000

Data Index: 1948701 (The Pile)

Max Activating Token Index: 589

Text #17

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.9056. Min Act: -0.0000

Data Index: 823219 (The Pile)

Max Activating Token Index: 746

Text #18

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.6632. Min Act: -0.0000

Data Index: 1663179 (The Pile)

Max Activating Token Index: 803

Text #19

Max Range: 1.0080. Min Range: -1.0080

Max Act: 0.6974. Min Act: -0.0000

Data Index: 181784 (The Pile)

Max Activating Token Index: 572