Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2413 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.7291. Min Act: -0.0000

Data Index: 1598866 (Python Code)

Max Activating Token Index: 635

Text #1

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.7556. Min Act: -0.0000

Data Index: 1385165 (Python Code)

Max Activating Token Index: 873

Text #2

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.7878. Min Act: -0.0000

Data Index: 1543450 (Python Code)

Max Activating Token Index: 700

Text #3

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.7255. Min Act: -0.0000

Data Index: 1552461 (Python Code)

Max Activating Token Index: 791

Text #4

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.6609. Min Act: -0.0000

Data Index: 1547806 (Python Code)

Max Activating Token Index: 698

Text #5

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.6377. Min Act: -0.0000

Data Index: 1480334 (Python Code)

Max Activating Token Index: 692

Text #6

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.7739. Min Act: -0.0000

Data Index: 1369696 (Python Code)

Max Activating Token Index: 645

Text #7

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.7544. Min Act: -0.0000

Data Index: 1450856 (Python Code)

Max Activating Token Index: 445

Text #8

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.8323. Min Act: -0.0000

Data Index: 1643649 (Python Code)

Max Activating Token Index: 678

Text #9

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.6284. Min Act: -0.0000

Data Index: 1468739 (Python Code)

Max Activating Token Index: 639

Text #10

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.4733. Min Act: -0.0000

Data Index: 1429549 (Python Code)

Max Activating Token Index: 218

Text #11

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.7266. Min Act: -0.0000

Data Index: 1507051 (Python Code)

Max Activating Token Index: 656

Text #12

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.5763. Min Act: -0.0000

Data Index: 1631362 (Python Code)

Max Activating Token Index: 563

Text #13

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.6122. Min Act: -0.0000

Data Index: 1394455 (Python Code)

Max Activating Token Index: 887

Text #14

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.6407. Min Act: -0.0000

Data Index: 1505998 (Python Code)

Max Activating Token Index: 587

Text #15

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.5080. Min Act: -0.0000

Data Index: 1629306 (Python Code)

Max Activating Token Index: 365

Text #16

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.5545. Min Act: -0.0000

Data Index: 1613864 (Python Code)

Max Activating Token Index: 352

Text #17

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.6283. Min Act: -0.0000

Data Index: 1441567 (Python Code)

Max Activating Token Index: 87

Text #18

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.6381. Min Act: -0.0000

Data Index: 1542774 (Python Code)

Max Activating Token Index: 182

Text #19

Max Range: 1.8323. Min Range: -1.8323

Max Act: 1.5582. Min Act: -0.0000

Data Index: 1475333 (Python Code)

Max Activating Token Index: 467