Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3918 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.9664. Min Act: -0.0000

Data Index: 1592277 (The Pile)

Max Activating Token Index: 255

Text #1

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.9458. Min Act: -0.0000

Data Index: 1029740 (The Pile)

Max Activating Token Index: 345

Text #2

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.8192. Min Act: -0.0000

Data Index: 1167805 (The Pile)

Max Activating Token Index: 406

Text #3

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.8147. Min Act: -0.0000

Data Index: 1657075 (The Pile)

Max Activating Token Index: 248

Text #4

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.8007. Min Act: -0.0000

Data Index: 54972 (The Pile)

Max Activating Token Index: 331

Text #5

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.7785. Min Act: -0.0000

Data Index: 1927953 (The Pile)

Max Activating Token Index: 160

Text #6

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.7765. Min Act: -0.0000

Data Index: 940915 (The Pile)

Max Activating Token Index: 866

Text #7

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.7084. Min Act: -0.0000

Data Index: 575371 (The Pile)

Max Activating Token Index: 906

Text #8

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.8040. Min Act: -0.0000

Data Index: 1067139 (The Pile)

Max Activating Token Index: 150

Text #9

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.7234. Min Act: -0.0000

Data Index: 322959 (The Pile)

Max Activating Token Index: 974

Text #10

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.6738. Min Act: -0.0000

Data Index: 285364 (The Pile)

Max Activating Token Index: 628

Text #11

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.6826. Min Act: -0.0000

Data Index: 1153087 (The Pile)

Max Activating Token Index: 1013

Text #12

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.6957. Min Act: -0.0000

Data Index: 1610735 (The Pile)

Max Activating Token Index: 518

Text #13

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.6752. Min Act: -0.0000

Data Index: 642129 (The Pile)

Max Activating Token Index: 475

Text #14

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.6556. Min Act: -0.0000

Data Index: 647222 (The Pile)

Max Activating Token Index: 950

Text #15

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.7897. Min Act: -0.0000

Data Index: 836356 (The Pile)

Max Activating Token Index: 889

Text #16

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.6755. Min Act: -0.0000

Data Index: 33633 (The Pile)

Max Activating Token Index: 540

Text #17

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.6274. Min Act: -0.0000

Data Index: 559799 (The Pile)

Max Activating Token Index: 66

Text #18

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.6669. Min Act: -0.0000

Data Index: 557189 (The Pile)

Max Activating Token Index: 568

Text #19

Max Range: 0.9664. Min Range: -0.9664

Max Act: 0.6599. Min Act: -0.0000

Data Index: 286747 (The Pile)

Max Activating Token Index: 194