Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 667 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1918. Min Range: -1.1918

Max Act: 1.1918. Min Act: -0.0000

Data Index: 1885286 (The Pile)

Max Activating Token Index: 773

Text #1

Max Range: 1.1918. Min Range: -1.1918

Max Act: 1.1643. Min Act: -0.0000

Data Index: 1231604 (The Pile)

Max Activating Token Index: 569

Text #2

Max Range: 1.1918. Min Range: -1.1918

Max Act: 1.0633. Min Act: -0.0000

Data Index: 1778489 (The Pile)

Max Activating Token Index: 93

Text #3

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9703. Min Act: -0.0000

Data Index: 663228 (The Pile)

Max Activating Token Index: 762

Text #4

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9261. Min Act: -0.0000

Data Index: 1984031 (The Pile)

Max Activating Token Index: 924

Text #5

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9438. Min Act: -0.0000

Data Index: 747793 (The Pile)

Max Activating Token Index: 680

Text #6

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9318. Min Act: -0.0000

Data Index: 654630 (The Pile)

Max Activating Token Index: 971

Text #7

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9093. Min Act: -0.0000

Data Index: 1478799 (The Pile)

Max Activating Token Index: 773

Text #8

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9787. Min Act: -0.0000

Data Index: 450611 (The Pile)

Max Activating Token Index: 454

Text #9

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9628. Min Act: -0.0000

Data Index: 1215465 (The Pile)

Max Activating Token Index: 19

Text #10

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9588. Min Act: -0.0000

Data Index: 198739 (The Pile)

Max Activating Token Index: 393

Text #11

Max Range: 1.1918. Min Range: -1.1918

Max Act: 1.0227. Min Act: -0.0000

Data Index: 858923 (The Pile)

Max Activating Token Index: 458

Text #12

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9137. Min Act: -0.0000

Data Index: 674370 (The Pile)

Max Activating Token Index: 181

Text #13

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.8890. Min Act: -0.0000

Data Index: 1341493 (The Pile)

Max Activating Token Index: 634

Text #14

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9382. Min Act: -0.0000

Data Index: 430481 (The Pile)

Max Activating Token Index: 100

Text #15

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9301. Min Act: -0.0000

Data Index: 209618 (The Pile)

Max Activating Token Index: 182

Text #16

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9140. Min Act: -0.0000

Data Index: 958644 (The Pile)

Max Activating Token Index: 60

Text #17

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9659. Min Act: -0.0000

Data Index: 448766 (The Pile)

Max Activating Token Index: 283

Text #18

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.8988. Min Act: -0.0000

Data Index: 1279273 (The Pile)

Max Activating Token Index: 45

Text #19

Max Range: 1.1918. Min Range: -1.1918

Max Act: 0.9348. Min Act: -0.0000

Data Index: 1118529 (The Pile)

Max Activating Token Index: 419