Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 448 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.4841. Min Range: -3.4841

Max Act: 3.4841. Min Act: -0.0000

Data Index: 817720 (The Pile)

Max Activating Token Index: 889

Text #1

Max Range: 3.4841. Min Range: -3.4841

Max Act: 3.2305. Min Act: -0.0000

Data Index: 904447 (The Pile)

Max Activating Token Index: 239

Text #2

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.9374. Min Act: -0.0000

Data Index: 935855 (The Pile)

Max Activating Token Index: 950

Text #3

Max Range: 3.4841. Min Range: -3.4841

Max Act: 3.3317. Min Act: -0.0000

Data Index: 1308915 (The Pile)

Max Activating Token Index: 1000

Text #4

Max Range: 3.4841. Min Range: -3.4841

Max Act: 3.0637. Min Act: -0.0000

Data Index: 1259434 (The Pile)

Max Activating Token Index: 958

Text #5

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.7184. Min Act: -0.0000

Data Index: 1304365 (The Pile)

Max Activating Token Index: 503

Text #6

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.5570. Min Act: -0.0000

Data Index: 129775 (The Pile)

Max Activating Token Index: 1021

Text #7

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.6043. Min Act: -0.0000

Data Index: 576341 (The Pile)

Max Activating Token Index: 890

Text #8

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.4969. Min Act: -0.0000

Data Index: 560588 (The Pile)

Max Activating Token Index: 133

Text #9

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.5292. Min Act: -0.0000

Data Index: 1348809 (The Pile)

Max Activating Token Index: 783

Text #10

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.4191. Min Act: -0.0000

Data Index: 1827378 (The Pile)

Max Activating Token Index: 869

Text #11

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.4443. Min Act: -0.0000

Data Index: 1650484 (The Pile)

Max Activating Token Index: 320

Text #12

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.3355. Min Act: -0.0000

Data Index: 917693 (The Pile)

Max Activating Token Index: 331

Text #13

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.2429. Min Act: -0.0000

Data Index: 791234 (The Pile)

Max Activating Token Index: 919

Text #14

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.3395. Min Act: -0.0000

Data Index: 1583873 (The Pile)

Max Activating Token Index: 986

Text #15

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.3144. Min Act: -0.0000

Data Index: 1224071 (The Pile)

Max Activating Token Index: 839

Text #16

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.3830. Min Act: -0.0000

Data Index: 523938 (The Pile)

Max Activating Token Index: 871

Text #17

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.3697. Min Act: -0.0000

Data Index: 399383 (The Pile)

Max Activating Token Index: 505

Text #18

Max Range: 3.4841. Min Range: -3.4841

Max Act: 2.5697. Min Act: -0.0000

Data Index: 1797673 (The Pile)

Max Activating Token Index: 291

Text #19

Max Range: 3.4841. Min Range: -3.4841

Max Act: 1.9762. Min Act: -0.0000

Data Index: 929187 (The Pile)

Max Activating Token Index: 822