Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 812 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.8039. Min Act: -0.0000

Data Index: 1590029 (The Pile)

Max Activating Token Index: 661

Text #1

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.6428. Min Act: -0.0000

Data Index: 489765 (The Pile)

Max Activating Token Index: 891

Text #2

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.5664. Min Act: -0.0000

Data Index: 1455032 (The Pile)

Max Activating Token Index: 911

Text #3

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.5645. Min Act: -0.0000

Data Index: 749881 (The Pile)

Max Activating Token Index: 636

Text #4

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.5442. Min Act: -0.0000

Data Index: 1268814 (The Pile)

Max Activating Token Index: 44

Text #5

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.5387. Min Act: -0.0000

Data Index: 1657822 (The Pile)

Max Activating Token Index: 496

Text #6

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.5564. Min Act: -0.0000

Data Index: 407497 (The Pile)

Max Activating Token Index: 647

Text #7

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.4853. Min Act: -0.0000

Data Index: 919371 (The Pile)

Max Activating Token Index: 938

Text #8

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.5624. Min Act: -0.0000

Data Index: 1011159 (The Pile)

Max Activating Token Index: 959

Text #9

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.5262. Min Act: -0.0000

Data Index: 966768 (The Pile)

Max Activating Token Index: 724

Text #10

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.4909. Min Act: -0.0000

Data Index: 1542153 (The Pile)

Max Activating Token Index: 212

Text #11

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.5023. Min Act: -0.0000

Data Index: 714744 (The Pile)

Max Activating Token Index: 331

Text #12

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.4408. Min Act: -0.0000

Data Index: 1014214 (The Pile)

Max Activating Token Index: 112

Text #13

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.4551. Min Act: -0.0000

Data Index: 242613 (The Pile)

Max Activating Token Index: 660

Text #14

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.4350. Min Act: -0.0000

Data Index: 973530 (The Pile)

Max Activating Token Index: 317

Text #15

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.4031. Min Act: -0.0000

Data Index: 1425490 (The Pile)

Max Activating Token Index: 69

Text #16

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.4543. Min Act: -0.0000

Data Index: 681444 (The Pile)

Max Activating Token Index: 896

Text #17

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.3640. Min Act: -0.0000

Data Index: 1246341 (The Pile)

Max Activating Token Index: 732

Text #18

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.3944. Min Act: -0.0000

Data Index: 1885657 (The Pile)

Max Activating Token Index: 667

Text #19

Max Range: 0.8039. Min Range: -0.8039

Max Act: 0.3737. Min Act: -0.0000

Data Index: 1323077 (The Pile)

Max Activating Token Index: 857