Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 3986 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('solu-12l')

Click the toggle to see the full text!

Text #0

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.1946. Min Act: -0.0002

Data Index: 1583204 (Python Code)

Max Activating Token Index: 860

Text #1

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.1941. Min Act: -0.0002

Data Index: 1643274 (Python Code)

Max Activating Token Index: 632

Text #2

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.1159. Min Act: -0.0002

Data Index: 1554852 (Python Code)

Max Activating Token Index: 188

Text #3

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.1087. Min Act: -0.0002

Data Index: 1613651 (Python Code)

Max Activating Token Index: 785

Text #4

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.1019. Min Act: -0.0002

Data Index: 1371748 (Python Code)

Max Activating Token Index: 102

Text #5

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.1067. Min Act: -0.0002

Data Index: 1642070 (Python Code)

Max Activating Token Index: 140

Text #6

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.0977. Min Act: -0.0002

Data Index: 1391325 (Python Code)

Max Activating Token Index: 712

Text #7

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.0987. Min Act: -0.0002

Data Index: 1614612 (Python Code)

Max Activating Token Index: 122

Text #8

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.1085. Min Act: -0.0002

Data Index: 1518222 (Python Code)

Max Activating Token Index: 121

Text #9

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.0862. Min Act: -0.0002

Data Index: 1482138 (Python Code)

Max Activating Token Index: 177

Text #10

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.0826. Min Act: -0.0002

Data Index: 1596822 (Python Code)

Max Activating Token Index: 93

Text #11

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.0895. Min Act: -0.0002

Data Index: 1368707 (Python Code)

Max Activating Token Index: 1007

Text #12

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.1068. Min Act: -0.0002

Data Index: 1574739 (Python Code)

Max Activating Token Index: 216

Text #13

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.0948. Min Act: -0.0002

Data Index: 1611812 (Python Code)

Max Activating Token Index: 75

Text #14

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.0748. Min Act: -0.0002

Data Index: 1494435 (Python Code)

Max Activating Token Index: 299

Text #15

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.0757. Min Act: -0.0002

Data Index: 1533361 (Python Code)

Max Activating Token Index: 163

Text #16

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.0876. Min Act: -0.0002

Data Index: 1429991 (Python Code)

Max Activating Token Index: 260

Text #17

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.0735. Min Act: -0.0002

Data Index: 1392892 (Python Code)

Max Activating Token Index: 600

Text #18

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.0550. Min Act: -0.0002

Data Index: 1529257 (Python Code)

Max Activating Token Index: 162

Text #19

Max Range: 1.1946. Min Range: -1.1946

Max Act: 1.0959. Min Act: -0.0002

Data Index: 1656473 (Python Code)

Max Activating Token Index: 393