Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 606 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.6330. Min Act: -0.0000

Data Index: 1102226 (The Pile)

Max Activating Token Index: 754

Text #1

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.2697. Min Act: -0.0000

Data Index: 1034703 (The Pile)

Max Activating Token Index: 615

Text #2

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.4394. Min Act: -0.0000

Data Index: 1300636 (The Pile)

Max Activating Token Index: 508

Text #3

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.1653. Min Act: -0.0000

Data Index: 1160368 (The Pile)

Max Activating Token Index: 835

Text #4

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.5482. Min Act: -0.0000

Data Index: 1438048 (The Pile)

Max Activating Token Index: 944

Text #5

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.2596. Min Act: -0.0000

Data Index: 1476577 (The Pile)

Max Activating Token Index: 279

Text #6

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.2730. Min Act: -0.0000

Data Index: 136119 (The Pile)

Max Activating Token Index: 799

Text #7

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.1571. Min Act: -0.0000

Data Index: 70049 (The Pile)

Max Activating Token Index: 593

Text #8

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.0798. Min Act: -0.0000

Data Index: 211121 (The Pile)

Max Activating Token Index: 599

Text #9

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.1451. Min Act: -0.0000

Data Index: 853489 (The Pile)

Max Activating Token Index: 565

Text #10

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.1369. Min Act: -0.0000

Data Index: 548839 (The Pile)

Max Activating Token Index: 779

Text #11

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.0674. Min Act: -0.0000

Data Index: 378460 (The Pile)

Max Activating Token Index: 453

Text #12

Max Range: 1.6330. Min Range: -1.6330

Max Act: 0.9340. Min Act: -0.0000

Data Index: 558609 (The Pile)

Max Activating Token Index: 731

Text #13

Max Range: 1.6330. Min Range: -1.6330

Max Act: 1.0391. Min Act: -0.0000

Data Index: 742216 (The Pile)

Max Activating Token Index: 397

Text #14

Max Range: 1.6330. Min Range: -1.6330

Max Act: 0.9612. Min Act: -0.0000

Data Index: 212195 (The Pile)

Max Activating Token Index: 804

Text #15

Max Range: 1.6330. Min Range: -1.6330

Max Act: 0.9870. Min Act: -0.0000

Data Index: 1477655 (The Pile)

Max Activating Token Index: 870

Text #16

Max Range: 1.6330. Min Range: -1.6330

Max Act: 0.9808. Min Act: -0.0000

Data Index: 1773099 (The Pile)

Max Activating Token Index: 611

Text #17

Max Range: 1.6330. Min Range: -1.6330

Max Act: 0.8638. Min Act: -0.0000

Data Index: 552134 (The Pile)

Max Activating Token Index: 593

Text #18

Max Range: 1.6330. Min Range: -1.6330

Max Act: 0.9138. Min Act: -0.0000

Data Index: 1536324 (The Pile)

Max Activating Token Index: 925

Text #19

Max Range: 1.6330. Min Range: -1.6330

Max Act: 0.9565. Min Act: -0.0000

Data Index: 1863609 (The Pile)

Max Activating Token Index: 186