Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 4376 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.6555. Min Act: -0.0000

Data Index: 1386992 (Python Code)

Max Activating Token Index: 917

Text #1

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.3509. Min Act: -0.0000

Data Index: 1502688 (Python Code)

Max Activating Token Index: 310

Text #2

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.3953. Min Act: -0.0000

Data Index: 1363536 (Python Code)

Max Activating Token Index: 904

Text #3

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.2339. Min Act: -0.0000

Data Index: 1490674 (Python Code)

Max Activating Token Index: 880

Text #4

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.1611. Min Act: -0.0000

Data Index: 1561137 (Python Code)

Max Activating Token Index: 311

Text #5

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.2238. Min Act: -0.0000

Data Index: 1559332 (Python Code)

Max Activating Token Index: 987

Text #6

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.1975. Min Act: -0.0000

Data Index: 1598100 (Python Code)

Max Activating Token Index: 104

Text #7

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.2415. Min Act: -0.0000

Data Index: 1486132 (Python Code)

Max Activating Token Index: 627

Text #8

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.1571. Min Act: -0.0000

Data Index: 1552299 (Python Code)

Max Activating Token Index: 138

Text #9

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.1432. Min Act: -0.0000

Data Index: 1381502 (Python Code)

Max Activating Token Index: 884

Text #10

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.1379. Min Act: -0.0000

Data Index: 1600485 (Python Code)

Max Activating Token Index: 961

Text #11

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.0974. Min Act: -0.0000

Data Index: 1388510 (Python Code)

Max Activating Token Index: 98

Text #12

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.0928. Min Act: -0.0000

Data Index: 1654014 (Python Code)

Max Activating Token Index: 406

Text #13

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.1920. Min Act: -0.0000

Data Index: 1642929 (Python Code)

Max Activating Token Index: 777

Text #14

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.1213. Min Act: -0.0000

Data Index: 1436511 (Python Code)

Max Activating Token Index: 271

Text #15

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.1540. Min Act: -0.0000

Data Index: 1431000 (Python Code)

Max Activating Token Index: 921

Text #16

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.1204. Min Act: -0.0000

Data Index: 1410396 (Python Code)

Max Activating Token Index: 285

Text #17

Max Range: 1.6555. Min Range: -1.6555

Max Act: 0.9906. Min Act: -0.0000

Data Index: 1561382 (Python Code)

Max Activating Token Index: 831

Text #18

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.0341. Min Act: -0.0000

Data Index: 1397238 (Python Code)

Max Activating Token Index: 632

Text #19

Max Range: 1.6555. Min Range: -1.6555

Max Act: 1.1518. Min Act: -0.0000

Data Index: 1543926 (Python Code)

Max Activating Token Index: 872