Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4101 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 9.6829. Min Range: -9.6829

Max Act: 9.6829. Min Act: -0.0000

Data Index: 918840 (The Pile)

Max Activating Token Index: 855

Text #1

Max Range: 9.6829. Min Range: -9.6829

Max Act: 7.7447. Min Act: -0.0000

Data Index: 361392 (The Pile)

Max Activating Token Index: 741

Text #2

Max Range: 9.6829. Min Range: -9.6829

Max Act: 7.9955. Min Act: -0.0000

Data Index: 1026806 (The Pile)

Max Activating Token Index: 1015

Text #3

Max Range: 9.6829. Min Range: -9.6829

Max Act: 6.9992. Min Act: -0.0000

Data Index: 1238073 (The Pile)

Max Activating Token Index: 825

Text #4

Max Range: 9.6829. Min Range: -9.6829

Max Act: 7.2642. Min Act: -0.0000

Data Index: 47274 (The Pile)

Max Activating Token Index: 666

Text #5

Max Range: 9.6829. Min Range: -9.6829

Max Act: 7.3261. Min Act: -0.0000

Data Index: 1721162 (The Pile)

Max Activating Token Index: 342

Text #6

Max Range: 9.6829. Min Range: -9.6829

Max Act: 7.6551. Min Act: -0.0000

Data Index: 1338327 (The Pile)

Max Activating Token Index: 980

Text #7

Max Range: 9.6829. Min Range: -9.6829

Max Act: 7.0350. Min Act: -0.0000

Data Index: 576010 (The Pile)

Max Activating Token Index: 556

Text #8

Max Range: 9.6829. Min Range: -9.6829

Max Act: 6.6172. Min Act: -0.0000

Data Index: 319893 (The Pile)

Max Activating Token Index: 674

Text #9

Max Range: 9.6829. Min Range: -9.6829

Max Act: 6.6823. Min Act: -0.0000

Data Index: 817548 (The Pile)

Max Activating Token Index: 812

Text #10

Max Range: 9.6829. Min Range: -9.6829

Max Act: 6.4707. Min Act: -0.0000

Data Index: 773729 (The Pile)

Max Activating Token Index: 787

Text #11

Max Range: 9.6829. Min Range: -9.6829

Max Act: 6.5784. Min Act: -0.0000

Data Index: 826399 (The Pile)

Max Activating Token Index: 872

Text #12

Max Range: 9.6829. Min Range: -9.6829

Max Act: 6.7856. Min Act: -0.0000

Data Index: 746916 (The Pile)

Max Activating Token Index: 358

Text #13

Max Range: 9.6829. Min Range: -9.6829

Max Act: 6.5930. Min Act: -0.0000

Data Index: 508776 (The Pile)

Max Activating Token Index: 550

Text #14

Max Range: 9.6829. Min Range: -9.6829

Max Act: 6.1671. Min Act: -0.0000

Data Index: 126723 (The Pile)

Max Activating Token Index: 769

Text #15

Max Range: 9.6829. Min Range: -9.6829

Max Act: 6.7740. Min Act: -0.0000

Data Index: 1392260 (The Pile)

Max Activating Token Index: 913

Text #16

Max Range: 9.6829. Min Range: -9.6829

Max Act: 6.7738. Min Act: -0.0000

Data Index: 782474 (The Pile)

Max Activating Token Index: 412

Text #17

Max Range: 9.6829. Min Range: -9.6829

Max Act: 6.6817. Min Act: -0.0000

Data Index: 177251 (The Pile)

Max Activating Token Index: 923

Text #18

Max Range: 9.6829. Min Range: -9.6829

Max Act: 6.5221. Min Act: -0.0000

Data Index: 626269 (The Pile)

Max Activating Token Index: 989

Text #19

Max Range: 9.6829. Min Range: -9.6829

Max Act: 7.2434. Min Act: -0.0000

Data Index: 343798 (The Pile)

Max Activating Token Index: 1008