Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2475 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.9703. Min Act: -0.0000

Data Index: 1003949 (The Pile)

Max Activating Token Index: 554

Text #1

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.8345. Min Act: -0.0000

Data Index: 91926 (The Pile)

Max Activating Token Index: 578

Text #2

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.8570. Min Act: -0.0000

Data Index: 1070450 (The Pile)

Max Activating Token Index: 863

Text #3

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.7633. Min Act: -0.0000

Data Index: 1829315 (The Pile)

Max Activating Token Index: 934

Text #4

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.7208. Min Act: -0.0000

Data Index: 839334 (The Pile)

Max Activating Token Index: 898

Text #5

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.7256. Min Act: -0.0000

Data Index: 618608 (The Pile)

Max Activating Token Index: 527

Text #6

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.8117. Min Act: -0.0000

Data Index: 40045 (The Pile)

Max Activating Token Index: 534

Text #7

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.7676. Min Act: -0.0000

Data Index: 51588 (The Pile)

Max Activating Token Index: 192

Text #8

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.7693. Min Act: -0.0000

Data Index: 193178 (The Pile)

Max Activating Token Index: 302

Text #9

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.7608. Min Act: -0.0000

Data Index: 335807 (The Pile)

Max Activating Token Index: 374

Text #10

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.7343. Min Act: -0.0000

Data Index: 536854 (The Pile)

Max Activating Token Index: 221

Text #11

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.7368. Min Act: -0.0000

Data Index: 1099953 (The Pile)

Max Activating Token Index: 119

Text #12

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.7335. Min Act: 0.0000

Data Index: 1486688 (The Pile)

Max Activating Token Index: 570

Text #13

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.7170. Min Act: -0.0000

Data Index: 572158 (The Pile)

Max Activating Token Index: 765

Text #14

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.7063. Min Act: -0.0000

Data Index: 1533978 (The Pile)

Max Activating Token Index: 19

Text #15

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.6841. Min Act: -0.0000

Data Index: 1879110 (The Pile)

Max Activating Token Index: 198

Text #16

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.6971. Min Act: -0.0000

Data Index: 876012 (The Pile)

Max Activating Token Index: 560

Text #17

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.6963. Min Act: -0.0000

Data Index: 1889273 (The Pile)

Max Activating Token Index: 751

Text #18

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.6658. Min Act: -0.0000

Data Index: 1656961 (The Pile)

Max Activating Token Index: 522

Text #19

Max Range: 0.9703. Min Range: -0.9703

Max Act: 0.6966. Min Act: -0.0000

Data Index: 1807920 (The Pile)

Max Activating Token Index: 173