Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2914 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0954. Min Act: -0.0000

Data Index: 1961878 (The Pile)

Max Activating Token Index: 597

Text #1

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0985. Min Act: -0.0000

Data Index: 359006 (The Pile)

Max Activating Token Index: 555

Text #2

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0983. Min Act: -0.0000

Data Index: 1014437 (The Pile)

Max Activating Token Index: 194

Text #3

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0899. Min Act: -0.0000

Data Index: 1663200 (The Pile)

Max Activating Token Index: 587

Text #4

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0850. Min Act: -0.0000

Data Index: 1700480 (The Pile)

Max Activating Token Index: 978

Text #5

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0905. Min Act: -0.0000

Data Index: 632840 (The Pile)

Max Activating Token Index: 440

Text #6

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0806. Min Act: -0.0000

Data Index: 1300732 (The Pile)

Max Activating Token Index: 915

Text #7

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0870. Min Act: -0.0000

Data Index: 818301 (The Pile)

Max Activating Token Index: 541

Text #8

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0898. Min Act: -0.0000

Data Index: 1391020 (The Pile)

Max Activating Token Index: 229

Text #9

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0892. Min Act: -0.0000

Data Index: 1898600 (The Pile)

Max Activating Token Index: 915

Text #10

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0884. Min Act: -0.0000

Data Index: 1914619 (The Pile)

Max Activating Token Index: 176

Text #11

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0847. Min Act: -0.0000

Data Index: 148606 (The Pile)

Max Activating Token Index: 48

Text #12

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0863. Min Act: -0.0000

Data Index: 1051601 (The Pile)

Max Activating Token Index: 1017

Text #13

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0889. Min Act: -0.0000

Data Index: 1343037 (The Pile)

Max Activating Token Index: 769

Text #14

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0772. Min Act: -0.0000

Data Index: 189297 (The Pile)

Max Activating Token Index: 1005

Text #15

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0718. Min Act: -0.0000

Data Index: 379722 (The Pile)

Max Activating Token Index: 604

Text #16

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0794. Min Act: -0.0000

Data Index: 1538652 (The Pile)

Max Activating Token Index: 305

Text #17

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0857. Min Act: -0.0000

Data Index: 784968 (The Pile)

Max Activating Token Index: 779

Text #18

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0796. Min Act: -0.0000

Data Index: 86637 (The Pile)

Max Activating Token Index: 335

Text #19

Max Range: 0.0985. Min Range: -0.0985

Max Act: 0.0740. Min Act: -0.0000

Data Index: 354009 (The Pile)

Max Activating Token Index: 973