Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 3608 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.1317. Min Act: -0.0000

Data Index: 1399810 (Python Code)

Max Activating Token Index: 620

Text #1

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.1745. Min Act: -0.0000

Data Index: 1532727 (Python Code)

Max Activating Token Index: 740

Text #2

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.1373. Min Act: -0.0000

Data Index: 1504739 (Python Code)

Max Activating Token Index: 841

Text #3

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.1698. Min Act: -0.0000

Data Index: 1423545 (Python Code)

Max Activating Token Index: 495

Text #4

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.1729. Min Act: -0.0000

Data Index: 1422362 (Python Code)

Max Activating Token Index: 771

Text #5

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.1007. Min Act: -0.0000

Data Index: 1622554 (Python Code)

Max Activating Token Index: 1020

Text #6

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.1070. Min Act: -0.0000

Data Index: 1455570 (Python Code)

Max Activating Token Index: 869

Text #7

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.1106. Min Act: -0.0000

Data Index: 1655923 (Python Code)

Max Activating Token Index: 827

Text #8

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.1298. Min Act: -0.0000

Data Index: 1530225 (Python Code)

Max Activating Token Index: 863

Text #9

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.0099. Min Act: -0.0000

Data Index: 1425507 (Python Code)

Max Activating Token Index: 747

Text #10

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.0723. Min Act: -0.0000

Data Index: 1373404 (Python Code)

Max Activating Token Index: 851

Text #11

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.1120. Min Act: -0.0000

Data Index: 1609768 (Python Code)

Max Activating Token Index: 919

Text #12

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.1177. Min Act: -0.0000

Data Index: 1583247 (Python Code)

Max Activating Token Index: 190

Text #13

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.0757. Min Act: -0.0000

Data Index: 1495358 (Python Code)

Max Activating Token Index: 945

Text #14

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.0778. Min Act: -0.0000

Data Index: 1380039 (Python Code)

Max Activating Token Index: 772

Text #15

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.0123. Min Act: -0.0000

Data Index: 1593307 (Python Code)

Max Activating Token Index: 850

Text #16

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.0460. Min Act: -0.0000

Data Index: 1378207 (Python Code)

Max Activating Token Index: 947

Text #17

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.0374. Min Act: -0.0000

Data Index: 1621059 (Python Code)

Max Activating Token Index: 972

Text #18

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.0117. Min Act: -0.0000

Data Index: 1574595 (Python Code)

Max Activating Token Index: 539

Text #19

Max Range: 1.1745. Min Range: -1.1745

Max Act: 1.0373. Min Act: -0.0000

Data Index: 1386110 (Python Code)

Max Activating Token Index: 863