Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 855 in Layer 11

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.9379. Min Act: -0.0000

Data Index: 1561012 (Python Code)

Max Activating Token Index: 919

Text #1

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.9651. Min Act: -0.0000

Data Index: 1451345 (Python Code)

Max Activating Token Index: 721

Text #2

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.7834. Min Act: -0.0000

Data Index: 1536788 (Python Code)

Max Activating Token Index: 226

Text #3

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.7740. Min Act: -0.0000

Data Index: 1608708 (Python Code)

Max Activating Token Index: 468

Text #4

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.6724. Min Act: -0.0000

Data Index: 1514913 (Python Code)

Max Activating Token Index: 968

Text #5

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.6601. Min Act: -0.0000

Data Index: 1373698 (Python Code)

Max Activating Token Index: 626

Text #6

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.4681. Min Act: -0.0000

Data Index: 1411783 (Python Code)

Max Activating Token Index: 881

Text #7

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.7118. Min Act: -0.0000

Data Index: 1366315 (Python Code)

Max Activating Token Index: 86

Text #8

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.6565. Min Act: -0.0000

Data Index: 1512548 (Python Code)

Max Activating Token Index: 824

Text #9

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.7449. Min Act: -0.0000

Data Index: 1574402 (Python Code)

Max Activating Token Index: 347

Text #10

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.6296. Min Act: -0.0000

Data Index: 1402437 (Python Code)

Max Activating Token Index: 618

Text #11

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.7171. Min Act: -0.0000

Data Index: 1613561 (Python Code)

Max Activating Token Index: 744

Text #12

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.7185. Min Act: -0.0000

Data Index: 1630810 (Python Code)

Max Activating Token Index: 759

Text #13

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.5935. Min Act: -0.0000

Data Index: 1528095 (Python Code)

Max Activating Token Index: 241

Text #14

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.4560. Min Act: -0.0000

Data Index: 1617714 (Python Code)

Max Activating Token Index: 98

Text #15

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.5256. Min Act: -0.0000

Data Index: 1485637 (Python Code)

Max Activating Token Index: 367

Text #16

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.5202. Min Act: -0.0000

Data Index: 1611043 (Python Code)

Max Activating Token Index: 849

Text #17

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.6121. Min Act: -0.0000

Data Index: 1603777 (Python Code)

Max Activating Token Index: 24

Text #18

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.7510. Min Act: -0.0000

Data Index: 1434552 (Python Code)

Max Activating Token Index: 222

Text #19

Max Range: 1.9651. Min Range: -1.9651

Max Act: 1.4003. Min Act: -0.0000

Data Index: 1370689 (Python Code)

Max Activating Token Index: 293