Model: SoLU Model: 6 Layers, 3072 Neurons per Layer

Dataset: The Pile

Neuron 1314 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-6l-pile')

Text #0

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.5059. Min Act: -0.0000

Data Index: 1399363 (The Pile)

Max Activating Token Index: 537

Text #1

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.4772. Min Act: -0.0000

Data Index: 1723025 (The Pile)

Max Activating Token Index: 461

Text #2

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.4627. Min Act: -0.0000

Data Index: 1850320 (The Pile)

Max Activating Token Index: 414

Text #3

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.3904. Min Act: 0.0000

Data Index: 1494901 (The Pile)

Max Activating Token Index: 963

Text #4

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.4511. Min Act: -0.0000

Data Index: 1783122 (The Pile)

Max Activating Token Index: 850

Text #5

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.4290. Min Act: -0.0000

Data Index: 613071 (The Pile)

Max Activating Token Index: 993

Text #6

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.4475. Min Act: -0.0000

Data Index: 1636991 (The Pile)

Max Activating Token Index: 554

Text #7

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.4309. Min Act: -0.0000

Data Index: 1428542 (The Pile)

Max Activating Token Index: 879

Text #8

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.4032. Min Act: 0.0000

Data Index: 1768910 (The Pile)

Max Activating Token Index: 991

Text #9

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.4246. Min Act: 0.0000

Data Index: 1144434 (The Pile)

Max Activating Token Index: 711

Text #10

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.4441. Min Act: -0.0000

Data Index: 978938 (The Pile)

Max Activating Token Index: 404

Text #11

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.4122. Min Act: -0.0000

Data Index: 621722 (The Pile)

Max Activating Token Index: 600

Text #12

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.4104. Min Act: -0.0000

Data Index: 189444 (The Pile)

Max Activating Token Index: 769

Text #13

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.3999. Min Act: -0.0000

Data Index: 1064557 (The Pile)

Max Activating Token Index: 481

Text #14

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.3985. Min Act: 0.0000

Data Index: 1681939 (The Pile)

Max Activating Token Index: 613

Text #15

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.3935. Min Act: -0.0000

Data Index: 554201 (The Pile)

Max Activating Token Index: 772

Text #16

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.4092. Min Act: -0.0000

Data Index: 510895 (The Pile)

Max Activating Token Index: 642

Text #17

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.3407. Min Act: -0.0000

Data Index: 897459 (The Pile)

Max Activating Token Index: 986

Text #18

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.3838. Min Act: -0.0000

Data Index: 475709 (The Pile)

Max Activating Token Index: 983

Text #19

Max Range: 0.5059. Min Range: -0.5059

Max Act: 0.3890. Min Act: -0.0000

Data Index: 1990425 (The Pile)

Max Activating Token Index: 672