Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2901 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.7548. Min Act: -0.0000

Data Index: 1391106 (Python Code)

Max Activating Token Index: 672

Text #1

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.5671. Min Act: -0.0000

Data Index: 1509683 (Python Code)

Max Activating Token Index: 35

Text #2

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.4236. Min Act: -0.0000

Data Index: 1474717 (Python Code)

Max Activating Token Index: 535

Text #3

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.3644. Min Act: -0.0000

Data Index: 1436539 (Python Code)

Max Activating Token Index: 41

Text #4

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.3424. Min Act: -0.0000

Data Index: 1397636 (Python Code)

Max Activating Token Index: 23

Text #5

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.3558. Min Act: -0.0000

Data Index: 1507858 (Python Code)

Max Activating Token Index: 479

Text #6

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.3180. Min Act: -0.0000

Data Index: 1477047 (Python Code)

Max Activating Token Index: 15

Text #7

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.1627. Min Act: -0.0000

Data Index: 1588085 (Python Code)

Max Activating Token Index: 645

Text #8

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.2732. Min Act: -0.0000

Data Index: 1579886 (Python Code)

Max Activating Token Index: 22

Text #9

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.2293. Min Act: -0.0000

Data Index: 1550539 (Python Code)

Max Activating Token Index: 68

Text #10

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.2780. Min Act: -0.0000

Data Index: 1604490 (Python Code)

Max Activating Token Index: 24

Text #11

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.2698. Min Act: -0.0000

Data Index: 1548329 (Python Code)

Max Activating Token Index: 18

Text #12

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.1864. Min Act: -0.0000

Data Index: 1627642 (Python Code)

Max Activating Token Index: 794

Text #13

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.3361. Min Act: -0.0000

Data Index: 1401796 (Python Code)

Max Activating Token Index: 948

Text #14

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.2957. Min Act: -0.0000

Data Index: 1606496 (Python Code)

Max Activating Token Index: 891

Text #15

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.2547. Min Act: -0.0000

Data Index: 1428793 (Python Code)

Max Activating Token Index: 568

Text #16

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.2539. Min Act: -0.0000

Data Index: 1585015 (Python Code)

Max Activating Token Index: 34

Text #17

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.2325. Min Act: -0.0000

Data Index: 1471224 (Python Code)

Max Activating Token Index: 138

Text #18

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.2767. Min Act: -0.0000

Data Index: 1604186 (Python Code)

Max Activating Token Index: 47

Text #19

Max Range: 1.7548. Min Range: -1.7548

Max Act: 1.2688. Min Act: -0.0000

Data Index: 1643729 (Python Code)

Max Activating Token Index: 68