Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1930 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5453. Min Range: -1.5453

Max Act: 1.5453. Min Act: -0.0000

Data Index: 21486 (The Pile)

Max Activating Token Index: 48

Text #1

Max Range: 1.5453. Min Range: -1.5453

Max Act: 1.3180. Min Act: -0.0000

Data Index: 323173 (The Pile)

Max Activating Token Index: 155

Text #2

Max Range: 1.5453. Min Range: -1.5453

Max Act: 1.1615. Min Act: -0.0000

Data Index: 1582559 (The Pile)

Max Activating Token Index: 529

Text #3

Max Range: 1.5453. Min Range: -1.5453

Max Act: 1.1956. Min Act: -0.0000

Data Index: 883118 (The Pile)

Max Activating Token Index: 968

Text #4

Max Range: 1.5453. Min Range: -1.5453

Max Act: 1.0995. Min Act: -0.0000

Data Index: 1791294 (The Pile)

Max Activating Token Index: 913

Text #5

Max Range: 1.5453. Min Range: -1.5453

Max Act: 1.0783. Min Act: -0.0000

Data Index: 1609081 (The Pile)

Max Activating Token Index: 428

Text #6

Max Range: 1.5453. Min Range: -1.5453

Max Act: 1.0671. Min Act: -0.0000

Data Index: 738701 (The Pile)

Max Activating Token Index: 430

Text #7

Max Range: 1.5453. Min Range: -1.5453

Max Act: 0.9863. Min Act: -0.0000

Data Index: 1632901 (The Pile)

Max Activating Token Index: 377

Text #8

Max Range: 1.5453. Min Range: -1.5453

Max Act: 0.9472. Min Act: -0.0000

Data Index: 1732116 (The Pile)

Max Activating Token Index: 724

Text #9

Max Range: 1.5453. Min Range: -1.5453

Max Act: 1.0078. Min Act: -0.0000

Data Index: 402170 (The Pile)

Max Activating Token Index: 953

Text #10

Max Range: 1.5453. Min Range: -1.5453

Max Act: 0.9505. Min Act: -0.0000

Data Index: 34627 (The Pile)

Max Activating Token Index: 932

Text #11

Max Range: 1.5453. Min Range: -1.5453

Max Act: 0.9816. Min Act: -0.0000

Data Index: 1846323 (The Pile)

Max Activating Token Index: 637

Text #12

Max Range: 1.5453. Min Range: -1.5453

Max Act: 0.9908. Min Act: -0.0000

Data Index: 217070 (The Pile)

Max Activating Token Index: 504

Text #13

Max Range: 1.5453. Min Range: -1.5453

Max Act: 0.9673. Min Act: -0.0000

Data Index: 223819 (The Pile)

Max Activating Token Index: 102

Text #14

Max Range: 1.5453. Min Range: -1.5453

Max Act: 0.9776. Min Act: -0.0000

Data Index: 1665705 (The Pile)

Max Activating Token Index: 47

Text #15

Max Range: 1.5453. Min Range: -1.5453

Max Act: 0.8706. Min Act: -0.0000

Data Index: 1628653 (The Pile)

Max Activating Token Index: 708

Text #16

Max Range: 1.5453. Min Range: -1.5453

Max Act: 0.9048. Min Act: -0.0000

Data Index: 339363 (The Pile)

Max Activating Token Index: 349

Text #17

Max Range: 1.5453. Min Range: -1.5453

Max Act: 0.9616. Min Act: -0.0000

Data Index: 1989717 (The Pile)

Max Activating Token Index: 678

Text #18

Max Range: 1.5453. Min Range: -1.5453

Max Act: 0.9278. Min Act: -0.0000

Data Index: 927684 (The Pile)

Max Activating Token Index: 45

Text #19

Max Range: 1.5453. Min Range: -1.5453

Max Act: 0.9308. Min Act: -0.0000

Data Index: 596329 (The Pile)

Max Activating Token Index: 1019