Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2968 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2963. Min Range: -1.2963

Max Act: 1.2963. Min Act: -0.0000

Data Index: 118602 (The Pile)

Max Activating Token Index: 227

Text #1

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.8783. Min Act: -0.0000

Data Index: 366691 (The Pile)

Max Activating Token Index: 848

Text #2

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.9387. Min Act: -0.0000

Data Index: 111760 (The Pile)

Max Activating Token Index: 373

Text #3

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.9315. Min Act: -0.0000

Data Index: 171318 (The Pile)

Max Activating Token Index: 449

Text #4

Max Range: 1.2963. Min Range: -1.2963

Max Act: 1.1067. Min Act: -0.0000

Data Index: 826694 (The Pile)

Max Activating Token Index: 704

Text #5

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.8397. Min Act: -0.0000

Data Index: 212243 (The Pile)

Max Activating Token Index: 716

Text #6

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.7831. Min Act: -0.0000

Data Index: 224313 (The Pile)

Max Activating Token Index: 784

Text #7

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.8245. Min Act: -0.0000

Data Index: 1489158 (The Pile)

Max Activating Token Index: 435

Text #8

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.7817. Min Act: -0.0000

Data Index: 1156225 (The Pile)

Max Activating Token Index: 393

Text #9

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.8402. Min Act: -0.0000

Data Index: 909855 (The Pile)

Max Activating Token Index: 546

Text #10

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.8056. Min Act: -0.0000

Data Index: 1266539 (The Pile)

Max Activating Token Index: 599

Text #11

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.8229. Min Act: -0.0000

Data Index: 1966041 (The Pile)

Max Activating Token Index: 343

Text #12

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.9250. Min Act: -0.0000

Data Index: 1232752 (The Pile)

Max Activating Token Index: 724

Text #13

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.6306. Min Act: -0.0000

Data Index: 760826 (The Pile)

Max Activating Token Index: 771

Text #14

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.7016. Min Act: -0.0000

Data Index: 830852 (The Pile)

Max Activating Token Index: 922

Text #15

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.7525. Min Act: -0.0000

Data Index: 660202 (The Pile)

Max Activating Token Index: 209

Text #16

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.7203. Min Act: -0.0000

Data Index: 468542 (The Pile)

Max Activating Token Index: 610

Text #17

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.7068. Min Act: -0.0000

Data Index: 1408999 (The Pile)

Max Activating Token Index: 534

Text #18

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.6249. Min Act: -0.0000

Data Index: 1204421 (The Pile)

Max Activating Token Index: 241

Text #19

Max Range: 1.2963. Min Range: -1.2963

Max Act: 0.6981. Min Act: -0.0000

Data Index: 1756406 (The Pile)

Max Activating Token Index: 342