Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2774 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.7058. Min Act: -0.0000

Data Index: 1753789 (The Pile)

Max Activating Token Index: 956

Text #1

Max Range: 4.2668. Min Range: -4.2668

Max Act: 4.2668. Min Act: -0.0000

Data Index: 937466 (The Pile)

Max Activating Token Index: 882

Text #2

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.9204. Min Act: -0.0000

Data Index: 1072516 (The Pile)

Max Activating Token Index: 508

Text #3

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.6062. Min Act: -0.0000

Data Index: 1825437 (The Pile)

Max Activating Token Index: 1011

Text #4

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.7935. Min Act: -0.0000

Data Index: 718675 (The Pile)

Max Activating Token Index: 996

Text #5

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.7798. Min Act: -0.0000

Data Index: 1994141 (The Pile)

Max Activating Token Index: 848

Text #6

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.5203. Min Act: -0.0000

Data Index: 774256 (The Pile)

Max Activating Token Index: 889

Text #7

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.5606. Min Act: -0.0000

Data Index: 445477 (The Pile)

Max Activating Token Index: 776

Text #8

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.9503. Min Act: -0.0000

Data Index: 1832490 (The Pile)

Max Activating Token Index: 942

Text #9

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.7393. Min Act: -0.0000

Data Index: 574542 (The Pile)

Max Activating Token Index: 810

Text #10

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.5759. Min Act: -0.0000

Data Index: 339581 (The Pile)

Max Activating Token Index: 713

Text #11

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.7119. Min Act: -0.0000

Data Index: 1581666 (The Pile)

Max Activating Token Index: 700

Text #12

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.6979. Min Act: -0.0000

Data Index: 1013073 (The Pile)

Max Activating Token Index: 718

Text #13

Max Range: 4.2668. Min Range: -4.2668

Max Act: 4.0248. Min Act: -0.0000

Data Index: 1281632 (The Pile)

Max Activating Token Index: 954

Text #14

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.6474. Min Act: -0.0000

Data Index: 1422473 (The Pile)

Max Activating Token Index: 823

Text #15

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.4891. Min Act: -0.0000

Data Index: 1462013 (The Pile)

Max Activating Token Index: 900

Text #16

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.6915. Min Act: -0.0000

Data Index: 380256 (The Pile)

Max Activating Token Index: 855

Text #17

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.9664. Min Act: -0.0000

Data Index: 208774 (The Pile)

Max Activating Token Index: 837

Text #18

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.3333. Min Act: -0.0000

Data Index: 725378 (The Pile)

Max Activating Token Index: 701

Text #19

Max Range: 4.2668. Min Range: -4.2668

Max Act: 3.9426. Min Act: -0.0000

Data Index: 1741961 (The Pile)

Max Activating Token Index: 783