Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 4281 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.5867. Min Act: -0.0000

Data Index: 1258944 (C4 (Web Text))

Max Activating Token Index: 87

Text #1

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.3953. Min Act: -0.0000

Data Index: 90495 (C4 (Web Text))

Max Activating Token Index: 844

Text #2

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.3861. Min Act: -0.0000

Data Index: 1188740 (C4 (Web Text))

Max Activating Token Index: 47

Text #3

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2929. Min Act: -0.0000

Data Index: 1141283 (C4 (Web Text))

Max Activating Token Index: 171

Text #4

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.3483. Min Act: -0.0000

Data Index: 192708 (C4 (Web Text))

Max Activating Token Index: 45

Text #5

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2860. Min Act: -0.0000

Data Index: 600009 (C4 (Web Text))

Max Activating Token Index: 882

Text #6

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.3092. Min Act: -0.0000

Data Index: 860756 (C4 (Web Text))

Max Activating Token Index: 302

Text #7

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2553. Min Act: -0.0000

Data Index: 325837 (C4 (Web Text))

Max Activating Token Index: 394

Text #8

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2494. Min Act: -0.0000

Data Index: 521561 (C4 (Web Text))

Max Activating Token Index: 310

Text #9

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2708. Min Act: -0.0000

Data Index: 737816 (C4 (Web Text))

Max Activating Token Index: 181

Text #10

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2312. Min Act: -0.0000

Data Index: 338722 (C4 (Web Text))

Max Activating Token Index: 745

Text #11

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2594. Min Act: -0.0000

Data Index: 813205 (C4 (Web Text))

Max Activating Token Index: 527

Text #12

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.3357. Min Act: -0.0000

Data Index: 559842 (C4 (Web Text))

Max Activating Token Index: 432

Text #13

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2537. Min Act: -0.0000

Data Index: 385349 (C4 (Web Text))

Max Activating Token Index: 752

Text #14

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2485. Min Act: -0.0000

Data Index: 280324 (C4 (Web Text))

Max Activating Token Index: 1018

Text #15

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2522. Min Act: -0.0000

Data Index: 389775 (C4 (Web Text))

Max Activating Token Index: 289

Text #16

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2062. Min Act: -0.0000

Data Index: 849142 (C4 (Web Text))

Max Activating Token Index: 469

Text #17

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2407. Min Act: -0.0000

Data Index: 1064058 (C4 (Web Text))

Max Activating Token Index: 634

Text #18

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2377. Min Act: -0.0000

Data Index: 929402 (C4 (Web Text))

Max Activating Token Index: 836

Text #19

Max Range: 1.5867. Min Range: -1.5867

Max Act: 1.2398. Min Act: -0.0000

Data Index: 232785 (C4 (Web Text))

Max Activating Token Index: 133