Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1235 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0859. Min Range: -1.0859

Max Act: 1.0859. Min Act: -0.0000

Data Index: 1440151 (The Pile)

Max Activating Token Index: 286

Text #1

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.9592. Min Act: -0.0000

Data Index: 98638 (The Pile)

Max Activating Token Index: 945

Text #2

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.9193. Min Act: -0.0000

Data Index: 1074102 (The Pile)

Max Activating Token Index: 866

Text #3

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.8993. Min Act: -0.0000

Data Index: 963514 (The Pile)

Max Activating Token Index: 603

Text #4

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.9330. Min Act: -0.0000

Data Index: 923860 (The Pile)

Max Activating Token Index: 539

Text #5

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.9280. Min Act: -0.0000

Data Index: 972289 (The Pile)

Max Activating Token Index: 524

Text #6

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.8814. Min Act: -0.0000

Data Index: 1266777 (The Pile)

Max Activating Token Index: 553

Text #7

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.9145. Min Act: -0.0000

Data Index: 1229782 (The Pile)

Max Activating Token Index: 167

Text #8

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.8824. Min Act: -0.0000

Data Index: 407572 (The Pile)

Max Activating Token Index: 912

Text #9

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.8520. Min Act: -0.0000

Data Index: 1870867 (The Pile)

Max Activating Token Index: 98

Text #10

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.8911. Min Act: -0.0000

Data Index: 1555343 (The Pile)

Max Activating Token Index: 385

Text #11

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.9362. Min Act: -0.0000

Data Index: 955303 (The Pile)

Max Activating Token Index: 879

Text #12

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.9599. Min Act: -0.0000

Data Index: 200838 (The Pile)

Max Activating Token Index: 674

Text #13

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.8225. Min Act: -0.0000

Data Index: 1911672 (The Pile)

Max Activating Token Index: 228

Text #14

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.8314. Min Act: -0.0000

Data Index: 249282 (The Pile)

Max Activating Token Index: 810

Text #15

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.8245. Min Act: -0.0000

Data Index: 442582 (The Pile)

Max Activating Token Index: 651

Text #16

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.7085. Min Act: -0.0000

Data Index: 228546 (The Pile)

Max Activating Token Index: 709

Text #17

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.8652. Min Act: -0.0000

Data Index: 1743481 (The Pile)

Max Activating Token Index: 844

Text #18

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.8193. Min Act: -0.0000

Data Index: 291331 (The Pile)

Max Activating Token Index: 312

Text #19

Max Range: 1.0859. Min Range: -1.0859

Max Act: 0.8697. Min Act: -0.0000

Data Index: 1924373 (The Pile)

Max Activating Token Index: 530