Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 417 in Layer 11

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 3.5802. Min Range: -3.5802

Max Act: 3.5367. Min Act: -0.0000

Data Index: 1578616 (Python Code)

Max Activating Token Index: 985

Text #1

Max Range: 3.5802. Min Range: -3.5802

Max Act: 3.5802. Min Act: -0.0000

Data Index: 1609076 (Python Code)

Max Activating Token Index: 196

Text #2

Max Range: 3.5802. Min Range: -3.5802

Max Act: 3.4164. Min Act: -0.0000

Data Index: 1385276 (Python Code)

Max Activating Token Index: 460

Text #3

Max Range: 3.5802. Min Range: -3.5802

Max Act: 3.2797. Min Act: -0.0000

Data Index: 1539827 (Python Code)

Max Activating Token Index: 750

Text #4

Max Range: 3.5802. Min Range: -3.5802

Max Act: 3.3957. Min Act: -0.0000

Data Index: 1380806 (Python Code)

Max Activating Token Index: 50

Text #5

Max Range: 3.5802. Min Range: -3.5802

Max Act: 3.1181. Min Act: -0.0000

Data Index: 391842 (C4 (Web Text))

Max Activating Token Index: 536

Text #6

Max Range: 3.5802. Min Range: -3.5802

Max Act: 3.3097. Min Act: -0.0000

Data Index: 1581553 (Python Code)

Max Activating Token Index: 280

Text #7

Max Range: 3.5802. Min Range: -3.5802

Max Act: 3.2668. Min Act: -0.0000

Data Index: 1468552 (Python Code)

Max Activating Token Index: 126

Text #8

Max Range: 3.5802. Min Range: -3.5802

Max Act: 3.1036. Min Act: -0.0000

Data Index: 1441471 (Python Code)

Max Activating Token Index: 618

Text #9

Max Range: 3.5802. Min Range: -3.5802

Max Act: 3.0755. Min Act: -0.0000

Data Index: 793511 (C4 (Web Text))

Max Activating Token Index: 1018

Text #10

Max Range: 3.5802. Min Range: -3.5802

Max Act: 3.0523. Min Act: 0.0000

Data Index: 807461 (C4 (Web Text))

Max Activating Token Index: 758

Text #11

Max Range: 3.5802. Min Range: -3.5802

Max Act: 3.0481. Min Act: -0.0000

Data Index: 1500696 (Python Code)

Max Activating Token Index: 19

Text #12

Max Range: 3.5802. Min Range: -3.5802

Max Act: 2.9901. Min Act: -0.0000

Data Index: 1371629 (Python Code)

Max Activating Token Index: 798

Text #13

Max Range: 3.5802. Min Range: -3.5802

Max Act: 2.9615. Min Act: -0.0000

Data Index: 1485199 (Python Code)

Max Activating Token Index: 328

Text #14

Max Range: 3.5802. Min Range: -3.5802

Max Act: 2.8069. Min Act: -0.0000

Data Index: 1597119 (Python Code)

Max Activating Token Index: 331

Text #15

Max Range: 3.5802. Min Range: -3.5802

Max Act: 2.8616. Min Act: -0.0000

Data Index: 694301 (C4 (Web Text))

Max Activating Token Index: 210

Text #16

Max Range: 3.5802. Min Range: -3.5802

Max Act: 2.7147. Min Act: -0.0000

Data Index: 1472986 (Python Code)

Max Activating Token Index: 204

Text #17

Max Range: 3.5802. Min Range: -3.5802

Max Act: 2.8358. Min Act: -0.0000

Data Index: 1583546 (Python Code)

Max Activating Token Index: 215

Text #18

Max Range: 3.5802. Min Range: -3.5802

Max Act: 2.7093. Min Act: -0.0000

Data Index: 99169 (C4 (Web Text))

Max Activating Token Index: 568

Text #19

Max Range: 3.5802. Min Range: -3.5802

Max Act: 2.6640. Min Act: -0.0000

Data Index: 1510663 (Python Code)

Max Activating Token Index: 97