Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2910 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0632. Min Act: -0.0000

Data Index: 675153 (C4 (Web Text))

Max Activating Token Index: 539

Text #1

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0556. Min Act: -0.0000

Data Index: 359343 (C4 (Web Text))

Max Activating Token Index: 320

Text #2

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0551. Min Act: -0.0000

Data Index: 403981 (C4 (Web Text))

Max Activating Token Index: 350

Text #3

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0522. Min Act: -0.0000

Data Index: 709700 (C4 (Web Text))

Max Activating Token Index: 176

Text #4

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0454. Min Act: -0.0000

Data Index: 1264011 (C4 (Web Text))

Max Activating Token Index: 38

Text #5

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0503. Min Act: -0.0000

Data Index: 1246510 (C4 (Web Text))

Max Activating Token Index: 336

Text #6

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0432. Min Act: -0.0000

Data Index: 403688 (C4 (Web Text))

Max Activating Token Index: 105

Text #7

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0435. Min Act: -0.0000

Data Index: 756940 (C4 (Web Text))

Max Activating Token Index: 216

Text #8

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0449. Min Act: -0.0000

Data Index: 650447 (C4 (Web Text))

Max Activating Token Index: 537

Text #9

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0417. Min Act: -0.0000

Data Index: 585832 (C4 (Web Text))

Max Activating Token Index: 868

Text #10

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0447. Min Act: -0.0000

Data Index: 1174467 (C4 (Web Text))

Max Activating Token Index: 874

Text #11

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0400. Min Act: -0.0000

Data Index: 1041281 (C4 (Web Text))

Max Activating Token Index: 410

Text #12

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0414. Min Act: -0.0000

Data Index: 240663 (C4 (Web Text))

Max Activating Token Index: 538

Text #13

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0418. Min Act: -0.0000

Data Index: 828217 (C4 (Web Text))

Max Activating Token Index: 821

Text #14

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0390. Min Act: -0.0000

Data Index: 1040168 (C4 (Web Text))

Max Activating Token Index: 673

Text #15

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0385. Min Act: -0.0000

Data Index: 158751 (C4 (Web Text))

Max Activating Token Index: 554

Text #16

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0391. Min Act: -0.0000

Data Index: 745566 (C4 (Web Text))

Max Activating Token Index: 343

Text #17

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0389. Min Act: -0.0000

Data Index: 918462 (C4 (Web Text))

Max Activating Token Index: 766

Text #18

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0375. Min Act: -0.0000

Data Index: 638978 (C4 (Web Text))

Max Activating Token Index: 814

Text #19

Max Range: 0.0632. Min Range: -0.0632

Max Act: 0.0388. Min Act: -0.0000

Data Index: 784430 (C4 (Web Text))

Max Activating Token Index: 639