Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4711 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.6716. Min Act: -0.0000

Data Index: 197561 (The Pile)

Max Activating Token Index: 625

Text #1

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.6673. Min Act: -0.0000

Data Index: 627453 (The Pile)

Max Activating Token Index: 384

Text #2

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.4831. Min Act: -0.0000

Data Index: 468313 (The Pile)

Max Activating Token Index: 759

Text #3

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.4115. Min Act: -0.0000

Data Index: 1865284 (The Pile)

Max Activating Token Index: 368

Text #4

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.3952. Min Act: -0.0000

Data Index: 460016 (The Pile)

Max Activating Token Index: 588

Text #5

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.4575. Min Act: -0.0000

Data Index: 1797400 (The Pile)

Max Activating Token Index: 195

Text #6

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.4532. Min Act: -0.0000

Data Index: 1465857 (The Pile)

Max Activating Token Index: 377

Text #7

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.4228. Min Act: -0.0000

Data Index: 158512 (The Pile)

Max Activating Token Index: 333

Text #8

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.5057. Min Act: -0.0000

Data Index: 1582851 (The Pile)

Max Activating Token Index: 472

Text #9

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.3212. Min Act: -0.0000

Data Index: 1308065 (The Pile)

Max Activating Token Index: 699

Text #10

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.4895. Min Act: -0.0000

Data Index: 1031715 (The Pile)

Max Activating Token Index: 480

Text #11

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.3646. Min Act: 0.0000

Data Index: 788858 (The Pile)

Max Activating Token Index: 631

Text #12

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.4623. Min Act: -0.0000

Data Index: 1194953 (The Pile)

Max Activating Token Index: 729

Text #13

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.3609. Min Act: -0.0000

Data Index: 783820 (The Pile)

Max Activating Token Index: 778

Text #14

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.3679. Min Act: -0.0000

Data Index: 1558390 (The Pile)

Max Activating Token Index: 368

Text #15

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.2833. Min Act: -0.0000

Data Index: 1587356 (The Pile)

Max Activating Token Index: 772

Text #16

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.2014. Min Act: -0.0000

Data Index: 853830 (The Pile)

Max Activating Token Index: 710

Text #17

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.3442. Min Act: -0.0000

Data Index: 1784927 (The Pile)

Max Activating Token Index: 674

Text #18

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.3187. Min Act: -0.0000

Data Index: 1390759 (The Pile)

Max Activating Token Index: 718

Text #19

Max Range: 1.6716. Min Range: -1.6716

Max Act: 1.1642. Min Act: -0.0000

Data Index: 1653514 (The Pile)

Max Activating Token Index: 94