Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 291 in Layer 10

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.4802. Min Act: -0.0000

Data Index: 1175447 (C4 (Web Text))

Max Activating Token Index: 162

Text #1

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.4511. Min Act: -0.0000

Data Index: 150801 (C4 (Web Text))

Max Activating Token Index: 357

Text #2

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.5517. Min Act: -0.0000

Data Index: 129714 (C4 (Web Text))

Max Activating Token Index: 590

Text #3

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.5637. Min Act: -0.0000

Data Index: 1125280 (C4 (Web Text))

Max Activating Token Index: 780

Text #4

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.4578. Min Act: -0.0000

Data Index: 937032 (C4 (Web Text))

Max Activating Token Index: 775

Text #5

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.3840. Min Act: -0.0000

Data Index: 660128 (C4 (Web Text))

Max Activating Token Index: 499

Text #6

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.3144. Min Act: -0.0000

Data Index: 716186 (C4 (Web Text))

Max Activating Token Index: 1007

Text #7

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.2511. Min Act: -0.0000

Data Index: 383913 (C4 (Web Text))

Max Activating Token Index: 591

Text #8

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.2645. Min Act: -0.0000

Data Index: 989691 (C4 (Web Text))

Max Activating Token Index: 422

Text #9

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.2300. Min Act: -0.0000

Data Index: 456497 (C4 (Web Text))

Max Activating Token Index: 79

Text #10

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.2235. Min Act: -0.0000

Data Index: 604738 (C4 (Web Text))

Max Activating Token Index: 150

Text #11

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.1752. Min Act: -0.0000

Data Index: 1076083 (C4 (Web Text))

Max Activating Token Index: 319

Text #12

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.1509. Min Act: -0.0000

Data Index: 1138114 (C4 (Web Text))

Max Activating Token Index: 441

Text #13

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.0806. Min Act: -0.0000

Data Index: 592437 (C4 (Web Text))

Max Activating Token Index: 480

Text #14

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.0443. Min Act: -0.0000

Data Index: 765161 (C4 (Web Text))

Max Activating Token Index: 321

Text #15

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.0143. Min Act: -0.0000

Data Index: 953179 (C4 (Web Text))

Max Activating Token Index: 532

Text #16

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.0013. Min Act: -0.0000

Data Index: 1199655 (C4 (Web Text))

Max Activating Token Index: 688

Text #17

Max Range: 1.5637. Min Range: -1.5637

Max Act: 1.0466. Min Act: -0.0000

Data Index: 268650 (C4 (Web Text))

Max Activating Token Index: 608

Text #18

Max Range: 1.5637. Min Range: -1.5637

Max Act: 0.9959. Min Act: -0.0000

Data Index: 620278 (C4 (Web Text))

Max Activating Token Index: 676

Text #19

Max Range: 1.5637. Min Range: -1.5637

Max Act: 0.9942. Min Act: -0.0000

Data Index: 1350781 (C4 (Web Text))

Max Activating Token Index: 221