Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2042 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 10.5752. Min Range: -10.5752

Max Act: 10.5752. Min Act: -0.0000

Data Index: 353377 (The Pile)

Max Activating Token Index: 355

Text #1

Max Range: 10.5752. Min Range: -10.5752

Max Act: 10.0325. Min Act: -0.0000

Data Index: 439355 (The Pile)

Max Activating Token Index: 512

Text #2

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.9761. Min Act: -0.0000

Data Index: 1258893 (The Pile)

Max Activating Token Index: 516

Text #3

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.7004. Min Act: -0.0000

Data Index: 911034 (The Pile)

Max Activating Token Index: 996

Text #4

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.8696. Min Act: -0.0000

Data Index: 216866 (The Pile)

Max Activating Token Index: 484

Text #5

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.5809. Min Act: -0.0000

Data Index: 460622 (The Pile)

Max Activating Token Index: 386

Text #6

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.7078. Min Act: -0.0000

Data Index: 1670203 (The Pile)

Max Activating Token Index: 494

Text #7

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.5475. Min Act: -0.0000

Data Index: 1851273 (The Pile)

Max Activating Token Index: 501

Text #8

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.9080. Min Act: -0.0000

Data Index: 1386250 (The Pile)

Max Activating Token Index: 302

Text #9

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.7934. Min Act: -0.0000

Data Index: 1421284 (The Pile)

Max Activating Token Index: 581

Text #10

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.4229. Min Act: -0.0000

Data Index: 847367 (The Pile)

Max Activating Token Index: 351

Text #11

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.9615. Min Act: -0.0000

Data Index: 535381 (The Pile)

Max Activating Token Index: 357

Text #12

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.7810. Min Act: -0.0000

Data Index: 479875 (The Pile)

Max Activating Token Index: 518

Text #13

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.4762. Min Act: -0.0000

Data Index: 661089 (The Pile)

Max Activating Token Index: 487

Text #14

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.7299. Min Act: -0.0000

Data Index: 1547937 (The Pile)

Max Activating Token Index: 396

Text #15

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.4525. Min Act: -0.0000

Data Index: 945045 (The Pile)

Max Activating Token Index: 349

Text #16

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.4874. Min Act: -0.0000

Data Index: 1042074 (The Pile)

Max Activating Token Index: 298

Text #17

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.4910. Min Act: -0.0000

Data Index: 688734 (The Pile)

Max Activating Token Index: 422

Text #18

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.3428. Min Act: -0.0000

Data Index: 1625643 (The Pile)

Max Activating Token Index: 560

Text #19

Max Range: 10.5752. Min Range: -10.5752

Max Act: 9.5642. Min Act: -0.0000

Data Index: 1791262 (The Pile)

Max Activating Token Index: 379