Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1806 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.8512. Min Act: -0.0000

Data Index: 1006670 (C4 (Web Text))

Max Activating Token Index: 1018

Text #1

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.6468. Min Act: -0.0000

Data Index: 321133 (C4 (Web Text))

Max Activating Token Index: 992

Text #2

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7594. Min Act: -0.0000

Data Index: 690074 (C4 (Web Text))

Max Activating Token Index: 1017

Text #3

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.5767. Min Act: -0.0000

Data Index: 435209 (C4 (Web Text))

Max Activating Token Index: 1020

Text #4

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.6656. Min Act: -0.0000

Data Index: 823307 (C4 (Web Text))

Max Activating Token Index: 682

Text #5

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.5683. Min Act: -0.0000

Data Index: 41673 (C4 (Web Text))

Max Activating Token Index: 805

Text #6

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.6554. Min Act: -0.0000

Data Index: 1323482 (C4 (Web Text))

Max Activating Token Index: 125

Text #7

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.6316. Min Act: -0.0000

Data Index: 1140769 (C4 (Web Text))

Max Activating Token Index: 923

Text #8

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.6362. Min Act: -0.0000

Data Index: 479644 (C4 (Web Text))

Max Activating Token Index: 927

Text #9

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.6223. Min Act: -0.0000

Data Index: 896686 (C4 (Web Text))

Max Activating Token Index: 925

Text #10

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.5556. Min Act: -0.0000

Data Index: 979477 (C4 (Web Text))

Max Activating Token Index: 751

Text #11

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.4311. Min Act: -0.0000

Data Index: 952689 (C4 (Web Text))

Max Activating Token Index: 714

Text #12

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.5763. Min Act: -0.0000

Data Index: 1075054 (C4 (Web Text))

Max Activating Token Index: 444

Text #13

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.5617. Min Act: -0.0000

Data Index: 890672 (C4 (Web Text))

Max Activating Token Index: 483

Text #14

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.6619. Min Act: -0.0000

Data Index: 1203248 (C4 (Web Text))

Max Activating Token Index: 398

Text #15

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.5546. Min Act: -0.0000

Data Index: 143785 (C4 (Web Text))

Max Activating Token Index: 866

Text #16

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.5383. Min Act: -0.0000

Data Index: 881848 (C4 (Web Text))

Max Activating Token Index: 201

Text #17

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.5549. Min Act: -0.0000

Data Index: 1004793 (C4 (Web Text))

Max Activating Token Index: 579

Text #18

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.4664. Min Act: -0.0000

Data Index: 596382 (C4 (Web Text))

Max Activating Token Index: 891

Text #19

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.5456. Min Act: -0.0000

Data Index: 353192 (C4 (Web Text))

Max Activating Token Index: 732