Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3173 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.6724. Min Act: -0.0000

Data Index: 1778227 (The Pile)

Max Activating Token Index: 587

Text #1

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4951. Min Act: -0.0000

Data Index: 1682030 (The Pile)

Max Activating Token Index: 270

Text #2

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.5037. Min Act: -0.0000

Data Index: 263980 (The Pile)

Max Activating Token Index: 135

Text #3

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.5046. Min Act: -0.0000

Data Index: 943278 (The Pile)

Max Activating Token Index: 411

Text #4

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4831. Min Act: -0.0000

Data Index: 610473 (The Pile)

Max Activating Token Index: 93

Text #5

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4469. Min Act: -0.0000

Data Index: 1084684 (The Pile)

Max Activating Token Index: 74

Text #6

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4488. Min Act: -0.0000

Data Index: 1801136 (The Pile)

Max Activating Token Index: 905

Text #7

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4620. Min Act: -0.0000

Data Index: 329561 (The Pile)

Max Activating Token Index: 306

Text #8

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4732. Min Act: -0.0000

Data Index: 170586 (The Pile)

Max Activating Token Index: 642

Text #9

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4318. Min Act: -0.0000

Data Index: 1339516 (The Pile)

Max Activating Token Index: 39

Text #10

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4415. Min Act: -0.0000

Data Index: 509516 (The Pile)

Max Activating Token Index: 134

Text #11

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4434. Min Act: -0.0000

Data Index: 1833019 (The Pile)

Max Activating Token Index: 297

Text #12

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4545. Min Act: -0.0000

Data Index: 1217134 (The Pile)

Max Activating Token Index: 11

Text #13

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4262. Min Act: -0.0000

Data Index: 738446 (The Pile)

Max Activating Token Index: 256

Text #14

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4151. Min Act: -0.0000

Data Index: 1108752 (The Pile)

Max Activating Token Index: 831

Text #15

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4349. Min Act: -0.0000

Data Index: 818468 (The Pile)

Max Activating Token Index: 31

Text #16

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4302. Min Act: -0.0000

Data Index: 1786723 (The Pile)

Max Activating Token Index: 569

Text #17

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4004. Min Act: -0.0000

Data Index: 1145702 (The Pile)

Max Activating Token Index: 87

Text #18

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4278. Min Act: -0.0000

Data Index: 51180 (The Pile)

Max Activating Token Index: 343

Text #19

Max Range: 0.6724. Min Range: -0.6724

Max Act: 0.4126. Min Act: -0.0000

Data Index: 446195 (The Pile)

Max Activating Token Index: 507