Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1313 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0882. Min Act: -0.0000

Data Index: 1559084 (The Pile)

Max Activating Token Index: 487

Text #1

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0848. Min Act: -0.0000

Data Index: 1723204 (The Pile)

Max Activating Token Index: 720

Text #2

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0903. Min Act: -0.0000

Data Index: 859656 (The Pile)

Max Activating Token Index: 202

Text #3

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0807. Min Act: -0.0000

Data Index: 253075 (The Pile)

Max Activating Token Index: 634

Text #4

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0852. Min Act: -0.0000

Data Index: 395687 (The Pile)

Max Activating Token Index: 58

Text #5

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0794. Min Act: -0.0000

Data Index: 320720 (The Pile)

Max Activating Token Index: 572

Text #6

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0813. Min Act: -0.0000

Data Index: 459811 (The Pile)

Max Activating Token Index: 955

Text #7

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0755. Min Act: -0.0000

Data Index: 1299050 (The Pile)

Max Activating Token Index: 785

Text #8

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0801. Min Act: -0.0000

Data Index: 1758222 (The Pile)

Max Activating Token Index: 965

Text #9

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0683. Min Act: -0.0000

Data Index: 516737 (The Pile)

Max Activating Token Index: 87

Text #10

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0728. Min Act: -0.0000

Data Index: 1752223 (The Pile)

Max Activating Token Index: 864

Text #11

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0748. Min Act: -0.0000

Data Index: 1150450 (The Pile)

Max Activating Token Index: 548

Text #12

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0709. Min Act: -0.0000

Data Index: 383717 (The Pile)

Max Activating Token Index: 683

Text #13

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0748. Min Act: -0.0000

Data Index: 813037 (The Pile)

Max Activating Token Index: 470

Text #14

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0720. Min Act: -0.0000

Data Index: 635583 (The Pile)

Max Activating Token Index: 422

Text #15

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0765. Min Act: -0.0000

Data Index: 1805149 (The Pile)

Max Activating Token Index: 295

Text #16

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0698. Min Act: -0.0000

Data Index: 353525 (The Pile)

Max Activating Token Index: 116

Text #17

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0751. Min Act: -0.0000

Data Index: 1511264 (The Pile)

Max Activating Token Index: 363

Text #18

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0739. Min Act: -0.0000

Data Index: 1348585 (The Pile)

Max Activating Token Index: 380

Text #19

Max Range: 0.0903. Min Range: -0.0903

Max Act: 0.0638. Min Act: -0.0000

Data Index: 864762 (The Pile)

Max Activating Token Index: 60