Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1892 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.6878. Min Act: -0.0000

Data Index: 1074252 (C4 (Web Text))

Max Activating Token Index: 1001

Text #1

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.3887. Min Act: -0.0000

Data Index: 1000283 (C4 (Web Text))

Max Activating Token Index: 403

Text #2

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.4379. Min Act: -0.0000

Data Index: 232541 (C4 (Web Text))

Max Activating Token Index: 310

Text #3

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.3038. Min Act: -0.0000

Data Index: 529924 (C4 (Web Text))

Max Activating Token Index: 887

Text #4

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.3050. Min Act: -0.0000

Data Index: 1062367 (C4 (Web Text))

Max Activating Token Index: 27

Text #5

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.3702. Min Act: -0.0000

Data Index: 854426 (C4 (Web Text))

Max Activating Token Index: 12

Text #6

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.5201. Min Act: -0.0000

Data Index: 1306119 (C4 (Web Text))

Max Activating Token Index: 207

Text #7

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.3139. Min Act: -0.0000

Data Index: 851706 (C4 (Web Text))

Max Activating Token Index: 672

Text #8

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.3640. Min Act: -0.0000

Data Index: 1146905 (C4 (Web Text))

Max Activating Token Index: 439

Text #9

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.2701. Min Act: -0.0000

Data Index: 1252809 (C4 (Web Text))

Max Activating Token Index: 956

Text #10

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.3246. Min Act: -0.0000

Data Index: 849252 (C4 (Web Text))

Max Activating Token Index: 967

Text #11

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.2284. Min Act: -0.0000

Data Index: 162343 (C4 (Web Text))

Max Activating Token Index: 99

Text #12

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.2959. Min Act: -0.0000

Data Index: 829184 (C4 (Web Text))

Max Activating Token Index: 396

Text #13

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.1587. Min Act: -0.0000

Data Index: 319246 (C4 (Web Text))

Max Activating Token Index: 517

Text #14

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.2326. Min Act: -0.0000

Data Index: 1332381 (C4 (Web Text))

Max Activating Token Index: 778

Text #15

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.1464. Min Act: -0.0000

Data Index: 286966 (C4 (Web Text))

Max Activating Token Index: 348

Text #16

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.3004. Min Act: -0.0000

Data Index: 515094 (C4 (Web Text))

Max Activating Token Index: 343

Text #17

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.1417. Min Act: -0.0000

Data Index: 927522 (C4 (Web Text))

Max Activating Token Index: 142

Text #18

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.1505. Min Act: -0.0000

Data Index: 1012495 (C4 (Web Text))

Max Activating Token Index: 798

Text #19

Max Range: 1.6878. Min Range: -1.6878

Max Act: 1.1300. Min Act: -0.0000

Data Index: 262430 (C4 (Web Text))

Max Activating Token Index: 973