Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1179 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.7165. Min Act: -0.0000

Data Index: 1412374 (Python Code)

Max Activating Token Index: 357

Text #1

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.6312. Min Act: -0.0001

Data Index: 1519503 (Python Code)

Max Activating Token Index: 146

Text #2

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.7071. Min Act: -0.0001

Data Index: 1630331 (Python Code)

Max Activating Token Index: 262

Text #3

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.6866. Min Act: -0.0000

Data Index: 1474845 (Python Code)

Max Activating Token Index: 477

Text #4

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.6100. Min Act: -0.0001

Data Index: 1504515 (Python Code)

Max Activating Token Index: 155

Text #5

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.5095. Min Act: -0.0000

Data Index: 1577447 (Python Code)

Max Activating Token Index: 344

Text #6

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.6013. Min Act: -0.0001

Data Index: 1379495 (Python Code)

Max Activating Token Index: 88

Text #7

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.5450. Min Act: -0.0001

Data Index: 1557998 (Python Code)

Max Activating Token Index: 141

Text #8

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.5935. Min Act: -0.0000

Data Index: 1450383 (Python Code)

Max Activating Token Index: 339

Text #9

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.5325. Min Act: -0.0001

Data Index: 1604503 (Python Code)

Max Activating Token Index: 51

Text #10

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.5122. Min Act: -0.0000

Data Index: 1470174 (Python Code)

Max Activating Token Index: 438

Text #11

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.5643. Min Act: -0.0001

Data Index: 1537371 (Python Code)

Max Activating Token Index: 126

Text #12

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.5566. Min Act: -0.0001

Data Index: 1435986 (Python Code)

Max Activating Token Index: 418

Text #13

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.4615. Min Act: -0.0000

Data Index: 1635386 (Python Code)

Max Activating Token Index: 244

Text #14

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.5026. Min Act: -0.0000

Data Index: 1633073 (Python Code)

Max Activating Token Index: 58

Text #15

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.4941. Min Act: -0.0000

Data Index: 1367669 (Python Code)

Max Activating Token Index: 844

Text #16

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.6706. Min Act: -0.0001

Data Index: 1384216 (Python Code)

Max Activating Token Index: 104

Text #17

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.5314. Min Act: -0.0000

Data Index: 1482945 (Python Code)

Max Activating Token Index: 246

Text #18

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.4038. Min Act: -0.0000

Data Index: 1590046 (Python Code)

Max Activating Token Index: 922

Text #19

Max Range: 1.7165. Min Range: -1.7165

Max Act: 1.3593. Min Act: -0.0000

Data Index: 1501586 (Python Code)

Max Activating Token Index: 221