Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 4528 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.8084. Min Act: -0.0002

Data Index: 1600135 (Python Code)

Max Activating Token Index: 181

Text #1

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.6905. Min Act: -0.0002

Data Index: 1512152 (Python Code)

Max Activating Token Index: 319

Text #2

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.8816. Min Act: -0.0002

Data Index: 1595306 (Python Code)

Max Activating Token Index: 885

Text #3

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.7899. Min Act: -0.0002

Data Index: 1567599 (Python Code)

Max Activating Token Index: 743

Text #4

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.8395. Min Act: -0.0002

Data Index: 1654704 (Python Code)

Max Activating Token Index: 93

Text #5

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.8347. Min Act: -0.0002

Data Index: 1402424 (Python Code)

Max Activating Token Index: 330

Text #6

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.5647. Min Act: -0.0002

Data Index: 1541597 (Python Code)

Max Activating Token Index: 231

Text #7

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.8233. Min Act: -0.0002

Data Index: 1422901 (Python Code)

Max Activating Token Index: 861

Text #8

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.7558. Min Act: -0.0002

Data Index: 1405352 (Python Code)

Max Activating Token Index: 43

Text #9

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.7828. Min Act: -0.0002

Data Index: 1404618 (Python Code)

Max Activating Token Index: 113

Text #10

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.6876. Min Act: -0.0002

Data Index: 1622229 (Python Code)

Max Activating Token Index: 408

Text #11

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.7554. Min Act: -0.0002

Data Index: 1636478 (Python Code)

Max Activating Token Index: 452

Text #12

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.7281. Min Act: -0.0002

Data Index: 1386672 (Python Code)

Max Activating Token Index: 301

Text #13

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.8063. Min Act: -0.0002

Data Index: 1545704 (Python Code)

Max Activating Token Index: 719

Text #14

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.8039. Min Act: -0.0002

Data Index: 1365890 (Python Code)

Max Activating Token Index: 385

Text #15

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.7136. Min Act: -0.0002

Data Index: 1421166 (Python Code)

Max Activating Token Index: 313

Text #16

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.5957. Min Act: -0.0002

Data Index: 1367231 (Python Code)

Max Activating Token Index: 425

Text #17

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.6046. Min Act: -0.0002

Data Index: 1521170 (Python Code)

Max Activating Token Index: 192

Text #18

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.7713. Min Act: -0.0002

Data Index: 1439123 (Python Code)

Max Activating Token Index: 131

Text #19

Max Range: 1.8816. Min Range: -1.8816

Max Act: 1.7687. Min Act: -0.0002

Data Index: 1479542 (Python Code)

Max Activating Token Index: 436