Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1332 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.1068. Min Act: -0.0000

Data Index: 1273068 (The Pile)

Max Activating Token Index: 1021

Text #1

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0916. Min Act: -0.0000

Data Index: 1879722 (The Pile)

Max Activating Token Index: 232

Text #2

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0859. Min Act: -0.0000

Data Index: 329629 (The Pile)

Max Activating Token Index: 751

Text #3

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0854. Min Act: -0.0000

Data Index: 1393897 (The Pile)

Max Activating Token Index: 1003

Text #4

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0856. Min Act: -0.0000

Data Index: 1356556 (The Pile)

Max Activating Token Index: 25

Text #5

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0843. Min Act: -0.0000

Data Index: 1032798 (The Pile)

Max Activating Token Index: 303

Text #6

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0813. Min Act: -0.0000

Data Index: 258093 (The Pile)

Max Activating Token Index: 400

Text #7

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0826. Min Act: -0.0000

Data Index: 384864 (The Pile)

Max Activating Token Index: 828

Text #8

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0814. Min Act: -0.0000

Data Index: 1036448 (The Pile)

Max Activating Token Index: 370

Text #9

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0721. Min Act: -0.0000

Data Index: 1290843 (The Pile)

Max Activating Token Index: 706

Text #10

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0730. Min Act: -0.0000

Data Index: 1867243 (The Pile)

Max Activating Token Index: 831

Text #11

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0728. Min Act: -0.0000

Data Index: 568079 (The Pile)

Max Activating Token Index: 585

Text #12

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0743. Min Act: -0.0000

Data Index: 95125 (The Pile)

Max Activating Token Index: 621

Text #13

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0707. Min Act: -0.0000

Data Index: 1544671 (The Pile)

Max Activating Token Index: 541

Text #14

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0653. Min Act: -0.0000

Data Index: 1865196 (The Pile)

Max Activating Token Index: 539

Text #15

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0709. Min Act: -0.0000

Data Index: 1384915 (The Pile)

Max Activating Token Index: 863

Text #16

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0699. Min Act: -0.0000

Data Index: 1030710 (The Pile)

Max Activating Token Index: 368

Text #17

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0723. Min Act: -0.0000

Data Index: 21261 (The Pile)

Max Activating Token Index: 926

Text #18

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0714. Min Act: -0.0000

Data Index: 127319 (The Pile)

Max Activating Token Index: 182

Text #19

Max Range: 0.1068. Min Range: -0.1068

Max Act: 0.0694. Min Act: -0.0000

Data Index: 1894825 (The Pile)

Max Activating Token Index: 678