Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 3443 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.7719. Min Act: -0.0000

Data Index: 1409110 (Python Code)

Max Activating Token Index: 995

Text #1

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.4305. Min Act: -0.0000

Data Index: 1634605 (Python Code)

Max Activating Token Index: 772

Text #2

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.3183. Min Act: -0.0000

Data Index: 1446101 (Python Code)

Max Activating Token Index: 111

Text #3

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.2873. Min Act: -0.0000

Data Index: 1524829 (Python Code)

Max Activating Token Index: 688

Text #4

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.2422. Min Act: -0.0000

Data Index: 1611896 (Python Code)

Max Activating Token Index: 520

Text #5

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.0297. Min Act: -0.0000

Data Index: 1482592 (Python Code)

Max Activating Token Index: 986

Text #6

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.1404. Min Act: -0.0000

Data Index: 1548711 (Python Code)

Max Activating Token Index: 885

Text #7

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.1215. Min Act: -0.0000

Data Index: 1435153 (Python Code)

Max Activating Token Index: 974

Text #8

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.1325. Min Act: -0.0000

Data Index: 1626623 (Python Code)

Max Activating Token Index: 950

Text #9

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.0929. Min Act: -0.0000

Data Index: 1600275 (Python Code)

Max Activating Token Index: 447

Text #10

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.0635. Min Act: -0.0000

Data Index: 1513643 (Python Code)

Max Activating Token Index: 744

Text #11

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.0117. Min Act: -0.0000

Data Index: 1547580 (Python Code)

Max Activating Token Index: 479

Text #12

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.0112. Min Act: -0.0000

Data Index: 1618957 (Python Code)

Max Activating Token Index: 621

Text #13

Max Range: 3.7719. Min Range: -3.7719

Max Act: 2.8808. Min Act: -0.0000

Data Index: 1547862 (Python Code)

Max Activating Token Index: 309

Text #14

Max Range: 3.7719. Min Range: -3.7719

Max Act: 2.8298. Min Act: -0.0000

Data Index: 1434478 (Python Code)

Max Activating Token Index: 1009

Text #15

Max Range: 3.7719. Min Range: -3.7719

Max Act: 2.9782. Min Act: -0.0000

Data Index: 1615807 (Python Code)

Max Activating Token Index: 117

Text #16

Max Range: 3.7719. Min Range: -3.7719

Max Act: 2.9636. Min Act: -0.0000

Data Index: 1385978 (Python Code)

Max Activating Token Index: 685

Text #17

Max Range: 3.7719. Min Range: -3.7719

Max Act: 3.2611. Min Act: -0.0000

Data Index: 1447917 (Python Code)

Max Activating Token Index: 757

Text #18

Max Range: 3.7719. Min Range: -3.7719

Max Act: 2.9607. Min Act: -0.0000

Data Index: 1443534 (Python Code)

Max Activating Token Index: 573

Text #19

Max Range: 3.7719. Min Range: -3.7719

Max Act: 2.7928. Min Act: -0.0000

Data Index: 1517032 (Python Code)

Max Activating Token Index: 1016