Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 3523 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.6765. Min Act: -0.0000

Data Index: 1380277 (Python Code)

Max Activating Token Index: 643

Text #1

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.7339. Min Act: -0.0000

Data Index: 1417764 (Python Code)

Max Activating Token Index: 502

Text #2

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.8584. Min Act: -0.0000

Data Index: 1544723 (Python Code)

Max Activating Token Index: 644

Text #3

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.7552. Min Act: -0.0000

Data Index: 1361454 (Python Code)

Max Activating Token Index: 268

Text #4

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.3613. Min Act: -0.0000

Data Index: 1545940 (Python Code)

Max Activating Token Index: 753

Text #5

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.5531. Min Act: -0.0000

Data Index: 1393380 (Python Code)

Max Activating Token Index: 161

Text #6

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.5190. Min Act: -0.0000

Data Index: 1633463 (Python Code)

Max Activating Token Index: 286

Text #7

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.1628. Min Act: -0.0000

Data Index: 1544653 (Python Code)

Max Activating Token Index: 1000

Text #8

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.4392. Min Act: -0.0000

Data Index: 1427214 (Python Code)

Max Activating Token Index: 88

Text #9

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.3555. Min Act: -0.0000

Data Index: 1467174 (Python Code)

Max Activating Token Index: 788

Text #10

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.5172. Min Act: -0.0000

Data Index: 1506562 (Python Code)

Max Activating Token Index: 325

Text #11

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.4663. Min Act: -0.0000

Data Index: 1481614 (Python Code)

Max Activating Token Index: 280

Text #12

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.4251. Min Act: -0.0000

Data Index: 998393 (C4 (Web Text))

Max Activating Token Index: 828

Text #13

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.1659. Min Act: -0.0000

Data Index: 1612605 (Python Code)

Max Activating Token Index: 966

Text #14

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.3326. Min Act: -0.0000

Data Index: 1377029 (Python Code)

Max Activating Token Index: 744

Text #15

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.5167. Min Act: -0.0000

Data Index: 1491445 (Python Code)

Max Activating Token Index: 537

Text #16

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.4580. Min Act: -0.0000

Data Index: 1416662 (Python Code)

Max Activating Token Index: 937

Text #17

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.2539. Min Act: -0.0000

Data Index: 1648318 (Python Code)

Max Activating Token Index: 411

Text #18

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.3754. Min Act: -0.0000

Data Index: 77921 (C4 (Web Text))

Max Activating Token Index: 957

Text #19

Max Range: 1.8584. Min Range: -1.8584

Max Act: 1.3646. Min Act: -0.0000

Data Index: 359635 (C4 (Web Text))

Max Activating Token Index: 994