Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 452 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.7444. Min Range: -5.7444

Max Act: 5.7444. Min Act: -0.0000

Data Index: 1542918 (The Pile)

Max Activating Token Index: 575

Text #1

Max Range: 5.7444. Min Range: -5.7444

Max Act: 4.2926. Min Act: -0.0000

Data Index: 115481 (The Pile)

Max Activating Token Index: 665

Text #2

Max Range: 5.7444. Min Range: -5.7444

Max Act: 4.3341. Min Act: -0.0000

Data Index: 76476 (The Pile)

Max Activating Token Index: 333

Text #3

Max Range: 5.7444. Min Range: -5.7444

Max Act: 4.7592. Min Act: -0.0000

Data Index: 1510901 (The Pile)

Max Activating Token Index: 988

Text #4

Max Range: 5.7444. Min Range: -5.7444

Max Act: 4.1200. Min Act: -0.0000

Data Index: 1751967 (The Pile)

Max Activating Token Index: 411

Text #5

Max Range: 5.7444. Min Range: -5.7444

Max Act: 4.1259. Min Act: -0.0000

Data Index: 1790492 (The Pile)

Max Activating Token Index: 258

Text #6

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.7580. Min Act: -0.0000

Data Index: 21156 (The Pile)

Max Activating Token Index: 847

Text #7

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.7475. Min Act: -0.0000

Data Index: 1551369 (The Pile)

Max Activating Token Index: 616

Text #8

Max Range: 5.7444. Min Range: -5.7444

Max Act: 4.0653. Min Act: -0.0000

Data Index: 1952732 (The Pile)

Max Activating Token Index: 615

Text #9

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.5773. Min Act: -0.0000

Data Index: 1420618 (The Pile)

Max Activating Token Index: 629

Text #10

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.5799. Min Act: -0.0000

Data Index: 1030987 (The Pile)

Max Activating Token Index: 736

Text #11

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.8658. Min Act: -0.0000

Data Index: 978695 (The Pile)

Max Activating Token Index: 132

Text #12

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.4976. Min Act: -0.0000

Data Index: 706829 (The Pile)

Max Activating Token Index: 758

Text #13

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.6429. Min Act: -0.0000

Data Index: 1069340 (The Pile)

Max Activating Token Index: 637

Text #14

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.9594. Min Act: -0.0000

Data Index: 1432475 (The Pile)

Max Activating Token Index: 851

Text #15

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.2883. Min Act: -0.0000

Data Index: 1469497 (The Pile)

Max Activating Token Index: 968

Text #16

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.6050. Min Act: -0.0000

Data Index: 490609 (The Pile)

Max Activating Token Index: 650

Text #17

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.5705. Min Act: -0.0000

Data Index: 211043 (The Pile)

Max Activating Token Index: 443

Text #18

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.2696. Min Act: -0.0000

Data Index: 1958206 (The Pile)

Max Activating Token Index: 1013

Text #19

Max Range: 5.7444. Min Range: -5.7444

Max Act: 3.3544. Min Act: -0.0000

Data Index: 1656999 (The Pile)

Max Activating Token Index: 924