Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 4825 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 5.8037. Min Range: -5.8037

Max Act: 5.6991. Min Act: -0.0000

Data Index: 1569892 (Python Code)

Max Activating Token Index: 546

Text #1

Max Range: 5.8037. Min Range: -5.8037

Max Act: 5.8037. Min Act: -0.0000

Data Index: 1473480 (Python Code)

Max Activating Token Index: 710

Text #2

Max Range: 5.8037. Min Range: -5.8037

Max Act: 5.3680. Min Act: -0.0000

Data Index: 1640792 (Python Code)

Max Activating Token Index: 863

Text #3

Max Range: 5.8037. Min Range: -5.8037

Max Act: 5.2809. Min Act: -0.0000

Data Index: 1417585 (Python Code)

Max Activating Token Index: 431

Text #4

Max Range: 5.8037. Min Range: -5.8037

Max Act: 5.0590. Min Act: -0.0000

Data Index: 1613665 (Python Code)

Max Activating Token Index: 704

Text #5

Max Range: 5.8037. Min Range: -5.8037

Max Act: 5.4253. Min Act: -0.0000

Data Index: 1502305 (Python Code)

Max Activating Token Index: 871

Text #6

Max Range: 5.8037. Min Range: -5.8037

Max Act: 4.5680. Min Act: -0.0000

Data Index: 1456217 (Python Code)

Max Activating Token Index: 723

Text #7

Max Range: 5.8037. Min Range: -5.8037

Max Act: 5.0720. Min Act: -0.0000

Data Index: 1601150 (Python Code)

Max Activating Token Index: 534

Text #8

Max Range: 5.8037. Min Range: -5.8037

Max Act: 5.5235. Min Act: -0.0000

Data Index: 1577011 (Python Code)

Max Activating Token Index: 862

Text #9

Max Range: 5.8037. Min Range: -5.8037

Max Act: 4.5517. Min Act: -0.0000

Data Index: 1476582 (Python Code)

Max Activating Token Index: 450

Text #10

Max Range: 5.8037. Min Range: -5.8037

Max Act: 5.0862. Min Act: -0.0000

Data Index: 1500310 (Python Code)

Max Activating Token Index: 541

Text #11

Max Range: 5.8037. Min Range: -5.8037

Max Act: 4.7219. Min Act: -0.0000

Data Index: 1633846 (Python Code)

Max Activating Token Index: 947

Text #12

Max Range: 5.8037. Min Range: -5.8037

Max Act: 5.0731. Min Act: -0.0000

Data Index: 1381707 (Python Code)

Max Activating Token Index: 486

Text #13

Max Range: 5.8037. Min Range: -5.8037

Max Act: 4.9924. Min Act: -0.0000

Data Index: 1448884 (Python Code)

Max Activating Token Index: 107

Text #14

Max Range: 5.8037. Min Range: -5.8037

Max Act: 5.0467. Min Act: -0.0000

Data Index: 1379919 (Python Code)

Max Activating Token Index: 732

Text #15

Max Range: 5.8037. Min Range: -5.8037

Max Act: 4.6897. Min Act: -0.0000

Data Index: 1641632 (Python Code)

Max Activating Token Index: 182

Text #16

Max Range: 5.8037. Min Range: -5.8037

Max Act: 4.6755. Min Act: -0.0000

Data Index: 1402543 (Python Code)

Max Activating Token Index: 995

Text #17

Max Range: 5.8037. Min Range: -5.8037

Max Act: 4.6700. Min Act: -0.0000

Data Index: 1582258 (Python Code)

Max Activating Token Index: 303

Text #18

Max Range: 5.8037. Min Range: -5.8037

Max Act: 4.7725. Min Act: -0.0000

Data Index: 1582251 (Python Code)

Max Activating Token Index: 553

Text #19

Max Range: 5.8037. Min Range: -5.8037

Max Act: 4.5414. Min Act: -0.0000

Data Index: 1473851 (Python Code)

Max Activating Token Index: 88