Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3521 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.9558. Min Act: -0.0000

Data Index: 1396868 (The Pile)

Max Activating Token Index: 355

Text #1

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.9710. Min Act: -0.0000

Data Index: 1725721 (The Pile)

Max Activating Token Index: 267

Text #2

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.7832. Min Act: -0.0000

Data Index: 1472272 (The Pile)

Max Activating Token Index: 489

Text #3

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.7520. Min Act: -0.0000

Data Index: 1527782 (The Pile)

Max Activating Token Index: 209

Text #4

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.6826. Min Act: -0.0000

Data Index: 433487 (The Pile)

Max Activating Token Index: 999

Text #5

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.7416. Min Act: -0.0000

Data Index: 1336050 (The Pile)

Max Activating Token Index: 755

Text #6

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.7552. Min Act: -0.0000

Data Index: 1933936 (The Pile)

Max Activating Token Index: 188

Text #7

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.7087. Min Act: -0.0000

Data Index: 1783853 (The Pile)

Max Activating Token Index: 93

Text #8

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.7486. Min Act: -0.0000

Data Index: 685798 (The Pile)

Max Activating Token Index: 301

Text #9

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.5921. Min Act: -0.0000

Data Index: 1467664 (The Pile)

Max Activating Token Index: 726

Text #10

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.7425. Min Act: -0.0000

Data Index: 252398 (The Pile)

Max Activating Token Index: 842

Text #11

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.7045. Min Act: -0.0000

Data Index: 1840710 (The Pile)

Max Activating Token Index: 992

Text #12

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.7059. Min Act: -0.0000

Data Index: 1132042 (The Pile)

Max Activating Token Index: 550

Text #13

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.7001. Min Act: -0.0000

Data Index: 374279 (The Pile)

Max Activating Token Index: 361

Text #14

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.6630. Min Act: -0.0000

Data Index: 739380 (The Pile)

Max Activating Token Index: 625

Text #15

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.6498. Min Act: -0.0000

Data Index: 530890 (The Pile)

Max Activating Token Index: 314

Text #16

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.6516. Min Act: -0.0000

Data Index: 1999372 (The Pile)

Max Activating Token Index: 644

Text #17

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.6624. Min Act: -0.0000

Data Index: 1552559 (The Pile)

Max Activating Token Index: 922

Text #18

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.6436. Min Act: -0.0000

Data Index: 1462363 (The Pile)

Max Activating Token Index: 137

Text #19

Max Range: 0.9710. Min Range: -0.9710

Max Act: 0.6059. Min Act: -0.0000

Data Index: 1996255 (The Pile)

Max Activating Token Index: 645