Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3326 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.7639. Min Act: -0.0000

Data Index: 452572 (The Pile)

Max Activating Token Index: 472

Text #1

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.6997. Min Act: -0.0000

Data Index: 10961 (The Pile)

Max Activating Token Index: 951

Text #2

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.6264. Min Act: -0.0000

Data Index: 1686544 (The Pile)

Max Activating Token Index: 141

Text #3

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.6143. Min Act: -0.0000

Data Index: 127975 (The Pile)

Max Activating Token Index: 143

Text #4

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.6333. Min Act: -0.0000

Data Index: 1811895 (The Pile)

Max Activating Token Index: 530

Text #5

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.6328. Min Act: -0.0000

Data Index: 1508798 (The Pile)

Max Activating Token Index: 375

Text #6

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.6292. Min Act: 0.0001

Data Index: 1010606 (The Pile)

Max Activating Token Index: 239

Text #7

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.6443. Min Act: -0.0000

Data Index: 63902 (The Pile)

Max Activating Token Index: 395

Text #8

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.6126. Min Act: -0.0000

Data Index: 705067 (The Pile)

Max Activating Token Index: 927

Text #9

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.6083. Min Act: -0.0000

Data Index: 654582 (The Pile)

Max Activating Token Index: 915

Text #10

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.5818. Min Act: -0.0000

Data Index: 1048421 (The Pile)

Max Activating Token Index: 944

Text #11

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.5722. Min Act: -0.0000

Data Index: 1127575 (The Pile)

Max Activating Token Index: 501

Text #12

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.5863. Min Act: 0.0000

Data Index: 1432478 (The Pile)

Max Activating Token Index: 411

Text #13

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.5705. Min Act: -0.0000

Data Index: 744773 (The Pile)

Max Activating Token Index: 735

Text #14

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.6070. Min Act: -0.0000

Data Index: 1199333 (The Pile)

Max Activating Token Index: 740

Text #15

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.5214. Min Act: -0.0000

Data Index: 385029 (The Pile)

Max Activating Token Index: 900

Text #16

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.6031. Min Act: -0.0000

Data Index: 197319 (The Pile)

Max Activating Token Index: 867

Text #17

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.5924. Min Act: 0.0000

Data Index: 93470 (The Pile)

Max Activating Token Index: 595

Text #18

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.5233. Min Act: -0.0000

Data Index: 436301 (The Pile)

Max Activating Token Index: 736

Text #19

Max Range: 0.7639. Min Range: -0.7639

Max Act: 0.5733. Min Act: -0.0000

Data Index: 84710 (The Pile)

Max Activating Token Index: 754