Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1347 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('solu-12l')

Click the toggle to see the full text!

Text #0

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1704. Min Act: -0.0002

Data Index: 1632372 (Python Code)

Max Activating Token Index: 689

Text #1

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1679. Min Act: -0.0001

Data Index: 1501435 (Python Code)

Max Activating Token Index: 152

Text #2

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1663. Min Act: -0.0001

Data Index: 1532921 (Python Code)

Max Activating Token Index: 79

Text #3

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1688. Min Act: -0.0001

Data Index: 1519155 (Python Code)

Max Activating Token Index: 163

Text #4

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1673. Min Act: -0.0001

Data Index: 1593993 (Python Code)

Max Activating Token Index: 138

Text #5

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1651. Min Act: -0.0001

Data Index: 1403886 (Python Code)

Max Activating Token Index: 244

Text #6

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1627. Min Act: -0.0001

Data Index: 1456782 (Python Code)

Max Activating Token Index: 1023

Text #7

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1641. Min Act: -0.0001

Data Index: 1450735 (Python Code)

Max Activating Token Index: 500

Text #8

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1613. Min Act: -0.0001

Data Index: 1439534 (Python Code)

Max Activating Token Index: 609

Text #9

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1610. Min Act: -0.0001

Data Index: 1547272 (Python Code)

Max Activating Token Index: 158

Text #10

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1605. Min Act: -0.0001

Data Index: 1481058 (Python Code)

Max Activating Token Index: 145

Text #11

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1614. Min Act: -0.0001

Data Index: 1573182 (Python Code)

Max Activating Token Index: 599

Text #12

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1595. Min Act: -0.0001

Data Index: 1576843 (Python Code)

Max Activating Token Index: 166

Text #13

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1606. Min Act: -0.0001

Data Index: 1434822 (Python Code)

Max Activating Token Index: 86

Text #14

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1602. Min Act: -0.0001

Data Index: 1622179 (Python Code)

Max Activating Token Index: 318

Text #15

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1617. Min Act: -0.0002

Data Index: 1593278 (Python Code)

Max Activating Token Index: 38

Text #16

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1581. Min Act: -0.0002

Data Index: 1566616 (Python Code)

Max Activating Token Index: 91

Text #17

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1589. Min Act: -0.0001

Data Index: 1485976 (Python Code)

Max Activating Token Index: 838

Text #18

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1615. Min Act: -0.0001

Data Index: 1362573 (Python Code)

Max Activating Token Index: 81

Text #19

Max Range: 0.1704. Min Range: -0.1704

Max Act: 0.1608. Min Act: -0.0001

Data Index: 1568108 (Python Code)

Max Activating Token Index: 808