Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 314 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.2884. Min Act: -0.0000

Data Index: 1555368 (Python Code)

Max Activating Token Index: 474

Text #1

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.2321. Min Act: -0.0000

Data Index: 1456406 (Python Code)

Max Activating Token Index: 881

Text #2

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.2169. Min Act: -0.0000

Data Index: 1393197 (Python Code)

Max Activating Token Index: 227

Text #3

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.1896. Min Act: -0.0000

Data Index: 1526023 (Python Code)

Max Activating Token Index: 571

Text #4

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.2844. Min Act: -0.0000

Data Index: 1568367 (Python Code)

Max Activating Token Index: 257

Text #5

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.1328. Min Act: -0.0000

Data Index: 1548790 (Python Code)

Max Activating Token Index: 941

Text #6

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.1950. Min Act: -0.0000

Data Index: 1582027 (Python Code)

Max Activating Token Index: 837

Text #7

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.1212. Min Act: -0.0000

Data Index: 1577342 (Python Code)

Max Activating Token Index: 662

Text #8

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.1172. Min Act: -0.0000

Data Index: 1609965 (Python Code)

Max Activating Token Index: 70

Text #9

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.1014. Min Act: -0.0000

Data Index: 1420864 (Python Code)

Max Activating Token Index: 841

Text #10

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.0891. Min Act: -0.0000

Data Index: 1589176 (Python Code)

Max Activating Token Index: 645

Text #11

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.0377. Min Act: -0.0000

Data Index: 1494210 (Python Code)

Max Activating Token Index: 718

Text #12

Max Range: 1.2884. Min Range: -1.2884

Max Act: 0.9941. Min Act: -0.0000

Data Index: 1364835 (Python Code)

Max Activating Token Index: 208

Text #13

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.0874. Min Act: -0.0000

Data Index: 1547787 (Python Code)

Max Activating Token Index: 439

Text #14

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.0577. Min Act: -0.0000

Data Index: 1619652 (Python Code)

Max Activating Token Index: 204

Text #15

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.0548. Min Act: -0.0000

Data Index: 1525002 (Python Code)

Max Activating Token Index: 888

Text #16

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.0366. Min Act: -0.0000

Data Index: 1523285 (Python Code)

Max Activating Token Index: 722

Text #17

Max Range: 1.2884. Min Range: -1.2884

Max Act: 0.9967. Min Act: -0.0000

Data Index: 1483278 (Python Code)

Max Activating Token Index: 955

Text #18

Max Range: 1.2884. Min Range: -1.2884

Max Act: 1.0240. Min Act: -0.0000

Data Index: 1501246 (Python Code)

Max Activating Token Index: 596

Text #19

Max Range: 1.2884. Min Range: -1.2884

Max Act: 0.9913. Min Act: -0.0000

Data Index: 1589259 (Python Code)

Max Activating Token Index: 973