Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1861 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 1.6235. Min Range: -1.6235

Max Act: 1.6235. Min Act: -0.0000

Data Index: 1561498 (Python Code)

Max Activating Token Index: 755

Text #1

Max Range: 1.6235. Min Range: -1.6235

Max Act: 1.5698. Min Act: -0.0000

Data Index: 1472653 (Python Code)

Max Activating Token Index: 479

Text #2

Max Range: 1.6235. Min Range: -1.6235

Max Act: 1.4808. Min Act: -0.0000

Data Index: 1646336 (Python Code)

Max Activating Token Index: 338

Text #3

Max Range: 1.6235. Min Range: -1.6235

Max Act: 1.2415. Min Act: -0.0000

Data Index: 1547406 (Python Code)

Max Activating Token Index: 167

Text #4

Max Range: 1.6235. Min Range: -1.6235

Max Act: 1.1968. Min Act: -0.0000

Data Index: 1375614 (Python Code)

Max Activating Token Index: 219

Text #5

Max Range: 1.6235. Min Range: -1.6235

Max Act: 1.2122. Min Act: -0.0000

Data Index: 1463299 (Python Code)

Max Activating Token Index: 394

Text #6

Max Range: 1.6235. Min Range: -1.6235

Max Act: 1.1400. Min Act: -0.0000

Data Index: 1541294 (Python Code)

Max Activating Token Index: 448

Text #7

Max Range: 1.6235. Min Range: -1.6235

Max Act: 1.0001. Min Act: -0.0000

Data Index: 1394940 (Python Code)

Max Activating Token Index: 902

Text #8

Max Range: 1.6235. Min Range: -1.6235

Max Act: 1.0731. Min Act: -0.0000

Data Index: 1625277 (Python Code)

Max Activating Token Index: 808

Text #9

Max Range: 1.6235. Min Range: -1.6235

Max Act: 1.2554. Min Act: -0.0000

Data Index: 1571811 (Python Code)

Max Activating Token Index: 305

Text #10

Max Range: 1.6235. Min Range: -1.6235

Max Act: 1.0706. Min Act: -0.0000

Data Index: 1443954 (Python Code)

Max Activating Token Index: 151

Text #11

Max Range: 1.6235. Min Range: -1.6235

Max Act: 0.9905. Min Act: -0.0000

Data Index: 1511674 (Python Code)

Max Activating Token Index: 145

Text #12

Max Range: 1.6235. Min Range: -1.6235

Max Act: 0.9061. Min Act: -0.0000

Data Index: 1452715 (Python Code)

Max Activating Token Index: 648

Text #13

Max Range: 1.6235. Min Range: -1.6235

Max Act: 0.9324. Min Act: -0.0000

Data Index: 1470027 (Python Code)

Max Activating Token Index: 886

Text #14

Max Range: 1.6235. Min Range: -1.6235

Max Act: 0.9422. Min Act: -0.0000

Data Index: 1387763 (Python Code)

Max Activating Token Index: 373

Text #15

Max Range: 1.6235. Min Range: -1.6235

Max Act: 0.9191. Min Act: -0.0000

Data Index: 1438686 (Python Code)

Max Activating Token Index: 605

Text #16

Max Range: 1.6235. Min Range: -1.6235

Max Act: 0.8984. Min Act: -0.0000

Data Index: 1430710 (Python Code)

Max Activating Token Index: 427

Text #17

Max Range: 1.6235. Min Range: -1.6235

Max Act: 0.8935. Min Act: -0.0000

Data Index: 1477815 (Python Code)

Max Activating Token Index: 397

Text #18

Max Range: 1.6235. Min Range: -1.6235

Max Act: 0.8367. Min Act: -0.0000

Data Index: 1432495 (Python Code)

Max Activating Token Index: 644

Text #19

Max Range: 1.6235. Min Range: -1.6235

Max Act: 0.7950. Min Act: -0.0000

Data Index: 1423447 (Python Code)

Max Activating Token Index: 329