Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 872 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1990. Min Act: -0.0000

Data Index: 766663 (The Pile)

Max Activating Token Index: 637

Text #1

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1918. Min Act: -0.0000

Data Index: 1851889 (The Pile)

Max Activating Token Index: 381

Text #2

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1968. Min Act: -0.0000

Data Index: 1025104 (The Pile)

Max Activating Token Index: 653

Text #3

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1804. Min Act: -0.0000

Data Index: 1003433 (The Pile)

Max Activating Token Index: 730

Text #4

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1854. Min Act: -0.0000

Data Index: 225932 (The Pile)

Max Activating Token Index: 1010

Text #5

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1754. Min Act: -0.0000

Data Index: 1757177 (The Pile)

Max Activating Token Index: 981

Text #6

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1697. Min Act: -0.0000

Data Index: 1170438 (The Pile)

Max Activating Token Index: 982

Text #7

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1550. Min Act: -0.0000

Data Index: 1763027 (The Pile)

Max Activating Token Index: 58

Text #8

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1456. Min Act: -0.0000

Data Index: 1520445 (The Pile)

Max Activating Token Index: 995

Text #9

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1376. Min Act: -0.0000

Data Index: 1671716 (The Pile)

Max Activating Token Index: 522

Text #10

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1437. Min Act: -0.0000

Data Index: 373766 (The Pile)

Max Activating Token Index: 1023

Text #11

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1593. Min Act: -0.0000

Data Index: 702177 (The Pile)

Max Activating Token Index: 783

Text #12

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1365. Min Act: -0.0000

Data Index: 1734363 (The Pile)

Max Activating Token Index: 739

Text #13

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1473. Min Act: -0.0000

Data Index: 1198798 (The Pile)

Max Activating Token Index: 744

Text #14

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1447. Min Act: -0.0000

Data Index: 277274 (The Pile)

Max Activating Token Index: 908

Text #15

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1656. Min Act: -0.0000

Data Index: 1904795 (The Pile)

Max Activating Token Index: 795

Text #16

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1497. Min Act: -0.0000

Data Index: 1337360 (The Pile)

Max Activating Token Index: 715

Text #17

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1324. Min Act: -0.0000

Data Index: 1223684 (The Pile)

Max Activating Token Index: 830

Text #18

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1363. Min Act: -0.0000

Data Index: 1158890 (The Pile)

Max Activating Token Index: 756

Text #19

Max Range: 0.1990. Min Range: -0.1990

Max Act: 0.1252. Min Act: -0.0000

Data Index: 32105 (The Pile)

Max Activating Token Index: 286