Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 166 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.9569. Min Act: -0.0000

Data Index: 1576151 (Python Code)

Max Activating Token Index: 25

Text #1

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.9565. Min Act: 0.0001

Data Index: 1457602 (Python Code)

Max Activating Token Index: 153

Text #2

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.9461. Min Act: 0.0000

Data Index: 1378984 (Python Code)

Max Activating Token Index: 78

Text #3

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.8050. Min Act: -0.0001

Data Index: 1597597 (Python Code)

Max Activating Token Index: 892

Text #4

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.6762. Min Act: 0.0001

Data Index: 1411784 (Python Code)

Max Activating Token Index: 202

Text #5

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.6732. Min Act: -0.0000

Data Index: 1429845 (Python Code)

Max Activating Token Index: 208

Text #6

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.6179. Min Act: 0.0001

Data Index: 1428424 (Python Code)

Max Activating Token Index: 129

Text #7

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.5813. Min Act: 0.0001

Data Index: 1482897 (Python Code)

Max Activating Token Index: 128

Text #8

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.5231. Min Act: 0.0000

Data Index: 1445419 (Python Code)

Max Activating Token Index: 52

Text #9

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.5982. Min Act: -0.0000

Data Index: 1380557 (Python Code)

Max Activating Token Index: 101

Text #10

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.5565. Min Act: -0.0000

Data Index: 1388519 (Python Code)

Max Activating Token Index: 512

Text #11

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.5896. Min Act: 0.0001

Data Index: 1564725 (Python Code)

Max Activating Token Index: 150

Text #12

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.5415. Min Act: -0.0000

Data Index: 1459135 (Python Code)

Max Activating Token Index: 244

Text #13

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.4790. Min Act: -0.0000

Data Index: 1632490 (Python Code)

Max Activating Token Index: 27

Text #14

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.4420. Min Act: -0.0000

Data Index: 1445810 (Python Code)

Max Activating Token Index: 191

Text #15

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.4780. Min Act: -0.0000

Data Index: 1436427 (Python Code)

Max Activating Token Index: 35

Text #16

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.5011. Min Act: -0.0000

Data Index: 1619638 (Python Code)

Max Activating Token Index: 935

Text #17

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.4172. Min Act: 0.0000

Data Index: 1444739 (Python Code)

Max Activating Token Index: 143

Text #18

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.5019. Min Act: -0.0000

Data Index: 1618990 (Python Code)

Max Activating Token Index: 17

Text #19

Max Range: 1.9569. Min Range: -1.9569

Max Act: 1.5030. Min Act: -0.0000

Data Index: 1552218 (Python Code)

Max Activating Token Index: 81