Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 543 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.7306. Min Act: -0.0000

Data Index: 643657 (The Pile)

Max Activating Token Index: 618

Text #1

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.7297. Min Act: -0.0000

Data Index: 530890 (The Pile)

Max Activating Token Index: 995

Text #2

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.6587. Min Act: -0.0000

Data Index: 1219915 (The Pile)

Max Activating Token Index: 811

Text #3

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.6866. Min Act: -0.0000

Data Index: 125697 (The Pile)

Max Activating Token Index: 493

Text #4

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.6365. Min Act: -0.0000

Data Index: 1927034 (The Pile)

Max Activating Token Index: 141

Text #5

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.6475. Min Act: -0.0000

Data Index: 473973 (The Pile)

Max Activating Token Index: 1000

Text #6

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.6294. Min Act: -0.0000

Data Index: 1199449 (The Pile)

Max Activating Token Index: 595

Text #7

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.5909. Min Act: -0.0000

Data Index: 263420 (The Pile)

Max Activating Token Index: 272

Text #8

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.6235. Min Act: -0.0000

Data Index: 1600751 (The Pile)

Max Activating Token Index: 567

Text #9

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.6115. Min Act: -0.0000

Data Index: 160343 (The Pile)

Max Activating Token Index: 958

Text #10

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.6075. Min Act: -0.0000

Data Index: 452337 (The Pile)

Max Activating Token Index: 918

Text #11

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.6237. Min Act: -0.0000

Data Index: 1033596 (The Pile)

Max Activating Token Index: 542

Text #12

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.5795. Min Act: -0.0000

Data Index: 1177128 (The Pile)

Max Activating Token Index: 209

Text #13

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.6158. Min Act: -0.0000

Data Index: 1942946 (The Pile)

Max Activating Token Index: 126

Text #14

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.5426. Min Act: -0.0000

Data Index: 1199728 (The Pile)

Max Activating Token Index: 979

Text #15

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.5463. Min Act: -0.0000

Data Index: 1610979 (The Pile)

Max Activating Token Index: 359

Text #16

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.5595. Min Act: -0.0000

Data Index: 1928438 (The Pile)

Max Activating Token Index: 353

Text #17

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.5726. Min Act: -0.0000

Data Index: 755753 (The Pile)

Max Activating Token Index: 113

Text #18

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.5537. Min Act: -0.0000

Data Index: 953372 (The Pile)

Max Activating Token Index: 891

Text #19

Max Range: 0.7306. Min Range: -0.7306

Max Act: 0.5330. Min Act: -0.0000

Data Index: 640572 (The Pile)

Max Activating Token Index: 753