Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2935 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 1.0290. Min Range: -1.0290

Max Act: 1.0290. Min Act: -0.0002

Data Index: 1460021 (Python Code)

Max Activating Token Index: 717

Text #1

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.8972. Min Act: -0.0003

Data Index: 1376081 (Python Code)

Max Activating Token Index: 862

Text #2

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.8396. Min Act: -0.0003

Data Index: 1413836 (Python Code)

Max Activating Token Index: 403

Text #3

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.8249. Min Act: -0.0003

Data Index: 1381563 (Python Code)

Max Activating Token Index: 157

Text #4

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7904. Min Act: -0.0003

Data Index: 1626889 (Python Code)

Max Activating Token Index: 311

Text #5

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7960. Min Act: -0.0003

Data Index: 1437524 (Python Code)

Max Activating Token Index: 859

Text #6

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7745. Min Act: -0.0003

Data Index: 1549190 (Python Code)

Max Activating Token Index: 142

Text #7

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7893. Min Act: -0.0003

Data Index: 1596929 (Python Code)

Max Activating Token Index: 237

Text #8

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7641. Min Act: -0.0003

Data Index: 1572053 (Python Code)

Max Activating Token Index: 66

Text #9

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7828. Min Act: -0.0003

Data Index: 1470971 (Python Code)

Max Activating Token Index: 146

Text #10

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7523. Min Act: -0.0003

Data Index: 1375070 (Python Code)

Max Activating Token Index: 359

Text #11

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7696. Min Act: -0.0003

Data Index: 1572483 (Python Code)

Max Activating Token Index: 150

Text #12

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7417. Min Act: -0.0003

Data Index: 1544940 (Python Code)

Max Activating Token Index: 98

Text #13

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7446. Min Act: -0.0002

Data Index: 1559209 (Python Code)

Max Activating Token Index: 192

Text #14

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7796. Min Act: -0.0003

Data Index: 1522898 (Python Code)

Max Activating Token Index: 213

Text #15

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7459. Min Act: -0.0003

Data Index: 1378639 (Python Code)

Max Activating Token Index: 499

Text #16

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7200. Min Act: -0.0003

Data Index: 1629284 (Python Code)

Max Activating Token Index: 899

Text #17

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7237. Min Act: -0.0003

Data Index: 1400107 (Python Code)

Max Activating Token Index: 231

Text #18

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7408. Min Act: -0.0003

Data Index: 1416157 (Python Code)

Max Activating Token Index: 187

Text #19

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.7362. Min Act: -0.0003

Data Index: 1448623 (Python Code)

Max Activating Token Index: 422