Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2677 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.5671. Min Act: -0.0000

Data Index: 1432990 (Python Code)

Max Activating Token Index: 254

Text #1

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.5231. Min Act: -0.0000

Data Index: 1624739 (Python Code)

Max Activating Token Index: 146

Text #2

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.4537. Min Act: -0.0000

Data Index: 1572874 (Python Code)

Max Activating Token Index: 175

Text #3

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.4317. Min Act: -0.0000

Data Index: 1405877 (Python Code)

Max Activating Token Index: 156

Text #4

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.4280. Min Act: -0.0000

Data Index: 1578808 (Python Code)

Max Activating Token Index: 876

Text #5

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.4250. Min Act: -0.0000

Data Index: 1640682 (Python Code)

Max Activating Token Index: 797

Text #6

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.4082. Min Act: -0.0000

Data Index: 1630436 (Python Code)

Max Activating Token Index: 870

Text #7

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.3834. Min Act: -0.0000

Data Index: 1401104 (Python Code)

Max Activating Token Index: 523

Text #8

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.2861. Min Act: -0.0000

Data Index: 1385164 (Python Code)

Max Activating Token Index: 977

Text #9

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.3446. Min Act: -0.0000

Data Index: 1398590 (Python Code)

Max Activating Token Index: 187

Text #10

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.3007. Min Act: -0.0000

Data Index: 1529046 (Python Code)

Max Activating Token Index: 88

Text #11

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.2966. Min Act: -0.0000

Data Index: 1368660 (Python Code)

Max Activating Token Index: 369

Text #12

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.2364. Min Act: -0.0000

Data Index: 1541427 (Python Code)

Max Activating Token Index: 897

Text #13

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.2655. Min Act: -0.0000

Data Index: 1557608 (Python Code)

Max Activating Token Index: 795

Text #14

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.2561. Min Act: -0.0000

Data Index: 1463350 (Python Code)

Max Activating Token Index: 595

Text #15

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.1967. Min Act: -0.0000

Data Index: 1653749 (Python Code)

Max Activating Token Index: 1012

Text #16

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.2269. Min Act: -0.0000

Data Index: 1645361 (Python Code)

Max Activating Token Index: 75

Text #17

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.1852. Min Act: -0.0000

Data Index: 1373658 (Python Code)

Max Activating Token Index: 68

Text #18

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.2199. Min Act: -0.0000

Data Index: 1508892 (Python Code)

Max Activating Token Index: 862

Text #19

Max Range: 1.5671. Min Range: -1.5671

Max Act: 1.1825. Min Act: -0.0000

Data Index: 1539152 (Python Code)

Max Activating Token Index: 144