Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2810 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('solu-12l')

Click the toggle to see the full text!

Text #0

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.5518. Min Act: -0.0002

Data Index: 213513 (C4 (Web Text))

Max Activating Token Index: 766

Text #1

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.4613. Min Act: -0.0002

Data Index: 858206 (C4 (Web Text))

Max Activating Token Index: 52

Text #2

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.4277. Min Act: -0.0002

Data Index: 923793 (C4 (Web Text))

Max Activating Token Index: 119

Text #3

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.4514. Min Act: -0.0002

Data Index: 798889 (C4 (Web Text))

Max Activating Token Index: 270

Text #4

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.3924. Min Act: -0.0002

Data Index: 1208066 (C4 (Web Text))

Max Activating Token Index: 175

Text #5

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.3878. Min Act: -0.0002

Data Index: 350190 (C4 (Web Text))

Max Activating Token Index: 32

Text #6

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.3983. Min Act: -0.0002

Data Index: 570078 (C4 (Web Text))

Max Activating Token Index: 280

Text #7

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.3552. Min Act: -0.0002

Data Index: 1042470 (C4 (Web Text))

Max Activating Token Index: 297

Text #8

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.3580. Min Act: -0.0002

Data Index: 1041547 (C4 (Web Text))

Max Activating Token Index: 401

Text #9

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.3426. Min Act: -0.0002

Data Index: 330181 (C4 (Web Text))

Max Activating Token Index: 851

Text #10

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.3487. Min Act: -0.0002

Data Index: 1112277 (C4 (Web Text))

Max Activating Token Index: 535

Text #11

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.3293. Min Act: -0.0002

Data Index: 940845 (C4 (Web Text))

Max Activating Token Index: 91

Text #12

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.2993. Min Act: -0.0002

Data Index: 858792 (C4 (Web Text))

Max Activating Token Index: 25

Text #13

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.3232. Min Act: -0.0002

Data Index: 896369 (C4 (Web Text))

Max Activating Token Index: 122

Text #14

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.3051. Min Act: -0.0002

Data Index: 1240981 (C4 (Web Text))

Max Activating Token Index: 32

Text #15

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.2950. Min Act: -0.0002

Data Index: 985154 (C4 (Web Text))

Max Activating Token Index: 366

Text #16

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.2737. Min Act: -0.0002

Data Index: 199624 (C4 (Web Text))

Max Activating Token Index: 879

Text #17

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.2721. Min Act: -0.0002

Data Index: 871219 (C4 (Web Text))

Max Activating Token Index: 424

Text #18

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.2966. Min Act: -0.0002

Data Index: 55672 (C4 (Web Text))

Max Activating Token Index: 86

Text #19

Max Range: 1.5518. Min Range: -1.5518

Max Act: 1.2681. Min Act: -0.0002

Data Index: 713544 (C4 (Web Text))

Max Activating Token Index: 215