Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1184 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l')

Text #0

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.1371. Min Act: -0.0001

Data Index: 1632455 (Python Code)

Max Activating Token Index: 52

Text #1

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.0943. Min Act: -0.0001

Data Index: 1498174 (Python Code)

Max Activating Token Index: 809

Text #2

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.1487. Min Act: -0.0001

Data Index: 1630661 (Python Code)

Max Activating Token Index: 361

Text #3

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.1046. Min Act: -0.0001

Data Index: 1386570 (Python Code)

Max Activating Token Index: 504

Text #4

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.0979. Min Act: -0.0001

Data Index: 851122 (C4 (Web Text))

Max Activating Token Index: 100

Text #5

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.0141. Min Act: -0.0001

Data Index: 1480428 (Python Code)

Max Activating Token Index: 250

Text #6

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.1110. Min Act: -0.0001

Data Index: 1585928 (Python Code)

Max Activating Token Index: 792

Text #7

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.0236. Min Act: -0.0001

Data Index: 1643544 (Python Code)

Max Activating Token Index: 100

Text #8

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.1474. Min Act: -0.0001

Data Index: 1446142 (Python Code)

Max Activating Token Index: 209

Text #9

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.0107. Min Act: -0.0001

Data Index: 1509256 (Python Code)

Max Activating Token Index: 153

Text #10

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.0727. Min Act: -0.0001

Data Index: 1492137 (Python Code)

Max Activating Token Index: 162

Text #11

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.0087. Min Act: -0.0001

Data Index: 1383211 (Python Code)

Max Activating Token Index: 73

Text #12

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.0334. Min Act: -0.0001

Data Index: 1603072 (Python Code)

Max Activating Token Index: 421

Text #13

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.0022. Min Act: -0.0001

Data Index: 1543533 (Python Code)

Max Activating Token Index: 52

Text #14

Max Range: 1.1487. Min Range: -1.1487

Max Act: 0.9882. Min Act: -0.0001

Data Index: 1474903 (Python Code)

Max Activating Token Index: 473

Text #15

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.0127. Min Act: -0.0001

Data Index: 1431957 (Python Code)

Max Activating Token Index: 183

Text #16

Max Range: 1.1487. Min Range: -1.1487

Max Act: 0.9829. Min Act: -0.0001

Data Index: 1491120 (Python Code)

Max Activating Token Index: 764

Text #17

Max Range: 1.1487. Min Range: -1.1487

Max Act: 1.0040. Min Act: -0.0001

Data Index: 1418594 (Python Code)

Max Activating Token Index: 814

Text #18

Max Range: 1.1487. Min Range: -1.1487

Max Act: 0.9705. Min Act: -0.0001

Data Index: 1376626 (Python Code)

Max Activating Token Index: 646

Text #19

Max Range: 1.1487. Min Range: -1.1487

Max Act: 0.9644. Min Act: -0.0001

Data Index: 1593257 (Python Code)

Max Activating Token Index: 126