Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 552 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.5703. Min Act: -0.0000

Data Index: 652976 (The Pile)

Max Activating Token Index: 237

Text #1

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.3888. Min Act: -0.0000

Data Index: 1932050 (The Pile)

Max Activating Token Index: 923

Text #2

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.2021. Min Act: -0.0000

Data Index: 110845 (The Pile)

Max Activating Token Index: 882

Text #3

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.3146. Min Act: -0.0000

Data Index: 1938173 (The Pile)

Max Activating Token Index: 824

Text #4

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.2847. Min Act: -0.0000

Data Index: 485534 (The Pile)

Max Activating Token Index: 841

Text #5

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.2375. Min Act: -0.0000

Data Index: 457952 (The Pile)

Max Activating Token Index: 326

Text #6

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.1593. Min Act: 0.0000

Data Index: 1927713 (The Pile)

Max Activating Token Index: 806

Text #7

Max Range: 5.5703. Min Range: -5.5703

Max Act: 4.9595. Min Act: -0.0000

Data Index: 826771 (The Pile)

Max Activating Token Index: 242

Text #8

Max Range: 5.5703. Min Range: -5.5703

Max Act: 4.7272. Min Act: -0.0000

Data Index: 1352521 (The Pile)

Max Activating Token Index: 923

Text #9

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.0771. Min Act: -0.0000

Data Index: 101114 (The Pile)

Max Activating Token Index: 326

Text #10

Max Range: 5.5703. Min Range: -5.5703

Max Act: 4.8822. Min Act: -0.0000

Data Index: 802153 (The Pile)

Max Activating Token Index: 787

Text #11

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.2502. Min Act: -0.0000

Data Index: 1254151 (The Pile)

Max Activating Token Index: 371

Text #12

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.0856. Min Act: -0.0000

Data Index: 1442526 (The Pile)

Max Activating Token Index: 590

Text #13

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.4000. Min Act: 0.0000

Data Index: 842385 (The Pile)

Max Activating Token Index: 1012

Text #14

Max Range: 5.5703. Min Range: -5.5703

Max Act: 4.7879. Min Act: -0.0000

Data Index: 1740237 (The Pile)

Max Activating Token Index: 864

Text #15

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.0744. Min Act: -0.0000

Data Index: 1345042 (The Pile)

Max Activating Token Index: 264

Text #16

Max Range: 5.5703. Min Range: -5.5703

Max Act: 4.8826. Min Act: -0.0000

Data Index: 968609 (The Pile)

Max Activating Token Index: 410

Text #17

Max Range: 5.5703. Min Range: -5.5703

Max Act: 5.2416. Min Act: -0.0000

Data Index: 361281 (The Pile)

Max Activating Token Index: 49

Text #18

Max Range: 5.5703. Min Range: -5.5703

Max Act: 4.8468. Min Act: -0.0000

Data Index: 1801859 (The Pile)

Max Activating Token Index: 469

Text #19

Max Range: 5.5703. Min Range: -5.5703

Max Act: 4.9127. Min Act: -0.0000

Data Index: 782175 (The Pile)

Max Activating Token Index: 100