Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 748 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 8.7698. Min Range: -8.7698

Max Act: 8.7698. Min Act: -0.0000

Data Index: 1602814 (Python Code)

Max Activating Token Index: 847

Text #1

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.8896. Min Act: -0.0000

Data Index: 1590713 (Python Code)

Max Activating Token Index: 652

Text #2

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.6103. Min Act: -0.0000

Data Index: 1640388 (Python Code)

Max Activating Token Index: 602

Text #3

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.5979. Min Act: -0.0000

Data Index: 1648806 (Python Code)

Max Activating Token Index: 844

Text #4

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.4958. Min Act: -0.0000

Data Index: 1440188 (Python Code)

Max Activating Token Index: 665

Text #5

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.6342. Min Act: -0.0000

Data Index: 1645783 (Python Code)

Max Activating Token Index: 569

Text #6

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.3848. Min Act: -0.0000

Data Index: 1384877 (Python Code)

Max Activating Token Index: 1018

Text #7

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.5200. Min Act: -0.0000

Data Index: 1521657 (Python Code)

Max Activating Token Index: 1014

Text #8

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.3557. Min Act: -0.0000

Data Index: 1386370 (Python Code)

Max Activating Token Index: 878

Text #9

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.3025. Min Act: -0.0000

Data Index: 1425184 (Python Code)

Max Activating Token Index: 713

Text #10

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.1977. Min Act: -0.0000

Data Index: 1458562 (Python Code)

Max Activating Token Index: 364

Text #11

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.1735. Min Act: -0.0000

Data Index: 1378704 (Python Code)

Max Activating Token Index: 660

Text #12

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.1521. Min Act: -0.0000

Data Index: 1458082 (Python Code)

Max Activating Token Index: 635

Text #13

Max Range: 8.7698. Min Range: -8.7698

Max Act: 6.7864. Min Act: -0.0000

Data Index: 1405403 (Python Code)

Max Activating Token Index: 108

Text #14

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.1278. Min Act: -0.0000

Data Index: 1490916 (Python Code)

Max Activating Token Index: 639

Text #15

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.0448. Min Act: -0.0000

Data Index: 1588192 (Python Code)

Max Activating Token Index: 331

Text #16

Max Range: 8.7698. Min Range: -8.7698

Max Act: 7.1408. Min Act: -0.0000

Data Index: 1641814 (Python Code)

Max Activating Token Index: 723

Text #17

Max Range: 8.7698. Min Range: -8.7698

Max Act: 6.9371. Min Act: -0.0000

Data Index: 1612417 (Python Code)

Max Activating Token Index: 692

Text #18

Max Range: 8.7698. Min Range: -8.7698

Max Act: 6.7303. Min Act: -0.0000

Data Index: 1549061 (Python Code)

Max Activating Token Index: 676

Text #19

Max Range: 8.7698. Min Range: -8.7698

Max Act: 6.8094. Min Act: -0.0000

Data Index: 1376892 (Python Code)

Max Activating Token Index: 95