Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1202 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.0644. Min Range: -3.0644

Max Act: 3.0644. Min Act: -0.0000

Data Index: 1671869 (The Pile)

Max Activating Token Index: 105

Text #1

Max Range: 3.0644. Min Range: -3.0644

Max Act: 2.7578. Min Act: -0.0000

Data Index: 1239486 (The Pile)

Max Activating Token Index: 290

Text #2

Max Range: 3.0644. Min Range: -3.0644

Max Act: 2.6739. Min Act: -0.0000

Data Index: 1553727 (The Pile)

Max Activating Token Index: 744

Text #3

Max Range: 3.0644. Min Range: -3.0644

Max Act: 2.4306. Min Act: -0.0000

Data Index: 1381832 (The Pile)

Max Activating Token Index: 404

Text #4

Max Range: 3.0644. Min Range: -3.0644

Max Act: 2.4108. Min Act: -0.0000

Data Index: 277052 (The Pile)

Max Activating Token Index: 891

Text #5

Max Range: 3.0644. Min Range: -3.0644

Max Act: 2.3805. Min Act: -0.0000

Data Index: 766475 (The Pile)

Max Activating Token Index: 526

Text #6

Max Range: 3.0644. Min Range: -3.0644

Max Act: 2.3706. Min Act: -0.0000

Data Index: 1457582 (The Pile)

Max Activating Token Index: 776

Text #7

Max Range: 3.0644. Min Range: -3.0644

Max Act: 2.0538. Min Act: -0.0000

Data Index: 86297 (The Pile)

Max Activating Token Index: 98

Text #8

Max Range: 3.0644. Min Range: -3.0644

Max Act: 2.2126. Min Act: -0.0000

Data Index: 1689263 (The Pile)

Max Activating Token Index: 501

Text #9

Max Range: 3.0644. Min Range: -3.0644

Max Act: 2.0581. Min Act: -0.0000

Data Index: 1090570 (The Pile)

Max Activating Token Index: 155

Text #10

Max Range: 3.0644. Min Range: -3.0644

Max Act: 2.1736. Min Act: 0.0000

Data Index: 1057566 (The Pile)

Max Activating Token Index: 773

Text #11

Max Range: 3.0644. Min Range: -3.0644

Max Act: 2.0742. Min Act: -0.0000

Data Index: 1610797 (The Pile)

Max Activating Token Index: 916

Text #12

Max Range: 3.0644. Min Range: -3.0644

Max Act: 2.0596. Min Act: -0.0000

Data Index: 1857864 (The Pile)

Max Activating Token Index: 57

Text #13

Max Range: 3.0644. Min Range: -3.0644

Max Act: 1.8533. Min Act: -0.0000

Data Index: 1495937 (The Pile)

Max Activating Token Index: 290

Text #14

Max Range: 3.0644. Min Range: -3.0644

Max Act: 1.6847. Min Act: -0.0000

Data Index: 1904110 (The Pile)

Max Activating Token Index: 188

Text #15

Max Range: 3.0644. Min Range: -3.0644

Max Act: 1.6790. Min Act: -0.0000

Data Index: 1618207 (The Pile)

Max Activating Token Index: 911

Text #16

Max Range: 3.0644. Min Range: -3.0644

Max Act: 1.6603. Min Act: -0.0000

Data Index: 94890 (The Pile)

Max Activating Token Index: 115

Text #17

Max Range: 3.0644. Min Range: -3.0644

Max Act: 1.6441. Min Act: -0.0000

Data Index: 1468210 (The Pile)

Max Activating Token Index: 172

Text #18

Max Range: 3.0644. Min Range: -3.0644

Max Act: 1.6667. Min Act: -0.0000

Data Index: 1761855 (The Pile)

Max Activating Token Index: 777

Text #19

Max Range: 3.0644. Min Range: -3.0644

Max Act: 1.7637. Min Act: -0.0000

Data Index: 1484619 (The Pile)

Max Activating Token Index: 140