Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 5993 in Layer 10

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.9396. Min Act: -0.0000

Data Index: 1634029 (Python Code)

Max Activating Token Index: 944

Text #1

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.7416. Min Act: -0.0000

Data Index: 1581202 (Python Code)

Max Activating Token Index: 936

Text #2

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.6914. Min Act: -0.0000

Data Index: 1448389 (Python Code)

Max Activating Token Index: 938

Text #3

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.8104. Min Act: -0.0000

Data Index: 1363704 (Python Code)

Max Activating Token Index: 919

Text #4

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.2623. Min Act: -0.0000

Data Index: 1480108 (Python Code)

Max Activating Token Index: 790

Text #5

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.5123. Min Act: -0.0000

Data Index: 1631108 (Python Code)

Max Activating Token Index: 517

Text #6

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.3205. Min Act: -0.0000

Data Index: 1640133 (Python Code)

Max Activating Token Index: 641

Text #7

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.1804. Min Act: -0.0000

Data Index: 1535141 (Python Code)

Max Activating Token Index: 627

Text #8

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.0895. Min Act: -0.0000

Data Index: 1504180 (Python Code)

Max Activating Token Index: 872

Text #9

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.0162. Min Act: -0.0000

Data Index: 1457104 (Python Code)

Max Activating Token Index: 847

Text #10

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.2997. Min Act: -0.0000

Data Index: 1641154 (Python Code)

Max Activating Token Index: 855

Text #11

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.0735. Min Act: -0.0000

Data Index: 1571771 (Python Code)

Max Activating Token Index: 59

Text #12

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.0898. Min Act: -0.0000

Data Index: 1577840 (Python Code)

Max Activating Token Index: 965

Text #13

Max Range: 3.9396. Min Range: -3.9396

Max Act: 2.8685. Min Act: -0.0000

Data Index: 1400059 (Python Code)

Max Activating Token Index: 751

Text #14

Max Range: 3.9396. Min Range: -3.9396

Max Act: 2.8569. Min Act: -0.0000

Data Index: 1452026 (Python Code)

Max Activating Token Index: 896

Text #15

Max Range: 3.9396. Min Range: -3.9396

Max Act: 3.1042. Min Act: -0.0000

Data Index: 1429940 (Python Code)

Max Activating Token Index: 787

Text #16

Max Range: 3.9396. Min Range: -3.9396

Max Act: 2.8085. Min Act: -0.0000

Data Index: 1586244 (Python Code)

Max Activating Token Index: 673

Text #17

Max Range: 3.9396. Min Range: -3.9396

Max Act: 2.8142. Min Act: -0.0000

Data Index: 1625522 (Python Code)

Max Activating Token Index: 354

Text #18

Max Range: 3.9396. Min Range: -3.9396

Max Act: 2.9537. Min Act: -0.0000

Data Index: 1616784 (Python Code)

Max Activating Token Index: 543

Text #19

Max Range: 3.9396. Min Range: -3.9396

Max Act: 2.7998. Min Act: -0.0000

Data Index: 1551263 (Python Code)

Max Activating Token Index: 42