Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4917 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0562. Min Range: -1.0562

Max Act: 1.0562. Min Act: -0.0000

Data Index: 416301 (The Pile)

Max Activating Token Index: 412

Text #1

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.8953. Min Act: -0.0000

Data Index: 1573549 (The Pile)

Max Activating Token Index: 629

Text #2

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.6334. Min Act: -0.0000

Data Index: 954605 (The Pile)

Max Activating Token Index: 771

Text #3

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.8071. Min Act: -0.0000

Data Index: 711386 (The Pile)

Max Activating Token Index: 871

Text #4

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.7526. Min Act: -0.0000

Data Index: 1747482 (The Pile)

Max Activating Token Index: 387

Text #5

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.7738. Min Act: -0.0000

Data Index: 130543 (The Pile)

Max Activating Token Index: 635

Text #6

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.6962. Min Act: -0.0000

Data Index: 75301 (The Pile)

Max Activating Token Index: 831

Text #7

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.7505. Min Act: -0.0000

Data Index: 1756849 (The Pile)

Max Activating Token Index: 377

Text #8

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.6616. Min Act: -0.0000

Data Index: 782278 (The Pile)

Max Activating Token Index: 976

Text #9

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.6222. Min Act: -0.0000

Data Index: 694355 (The Pile)

Max Activating Token Index: 747

Text #10

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.5255. Min Act: -0.0000

Data Index: 1253253 (The Pile)

Max Activating Token Index: 847

Text #11

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.6989. Min Act: -0.0000

Data Index: 720923 (The Pile)

Max Activating Token Index: 365

Text #12

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.7369. Min Act: -0.0000

Data Index: 87239 (The Pile)

Max Activating Token Index: 607

Text #13

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.7002. Min Act: -0.0000

Data Index: 1250676 (The Pile)

Max Activating Token Index: 761

Text #14

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.7109. Min Act: -0.0000

Data Index: 785175 (The Pile)

Max Activating Token Index: 339

Text #15

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.6053. Min Act: -0.0000

Data Index: 1791442 (The Pile)

Max Activating Token Index: 508

Text #16

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.6904. Min Act: -0.0000

Data Index: 1925268 (The Pile)

Max Activating Token Index: 771

Text #17

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.5763. Min Act: -0.0000

Data Index: 484002 (The Pile)

Max Activating Token Index: 564

Text #18

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.7473. Min Act: -0.0000

Data Index: 274677 (The Pile)

Max Activating Token Index: 543

Text #19

Max Range: 1.0562. Min Range: -1.0562

Max Act: 0.6305. Min Act: -0.0000

Data Index: 1034755 (The Pile)

Max Activating Token Index: 406