Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 465 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.9730. Min Act: -0.0000

Data Index: 53890 (The Pile)

Max Activating Token Index: 996

Text #1

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.7660. Min Act: -0.0000

Data Index: 67581 (The Pile)

Max Activating Token Index: 1010

Text #2

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.7323. Min Act: -0.0000

Data Index: 480843 (The Pile)

Max Activating Token Index: 904

Text #3

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.7637. Min Act: -0.0000

Data Index: 1023567 (The Pile)

Max Activating Token Index: 388

Text #4

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.7977. Min Act: -0.0000

Data Index: 787002 (The Pile)

Max Activating Token Index: 368

Text #5

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.7246. Min Act: -0.0000

Data Index: 271098 (The Pile)

Max Activating Token Index: 735

Text #6

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.7168. Min Act: -0.0000

Data Index: 1303586 (The Pile)

Max Activating Token Index: 458

Text #7

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.6646. Min Act: -0.0000

Data Index: 1439219 (The Pile)

Max Activating Token Index: 774

Text #8

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.7299. Min Act: -0.0000

Data Index: 1714686 (The Pile)

Max Activating Token Index: 522

Text #9

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.6735. Min Act: -0.0000

Data Index: 1341949 (The Pile)

Max Activating Token Index: 340

Text #10

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.6808. Min Act: -0.0000

Data Index: 266828 (The Pile)

Max Activating Token Index: 639

Text #11

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.6737. Min Act: -0.0000

Data Index: 693356 (The Pile)

Max Activating Token Index: 705

Text #12

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.6459. Min Act: -0.0000

Data Index: 219992 (The Pile)

Max Activating Token Index: 37

Text #13

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.6415. Min Act: -0.0000

Data Index: 793368 (The Pile)

Max Activating Token Index: 334

Text #14

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.6863. Min Act: -0.0000

Data Index: 39403 (The Pile)

Max Activating Token Index: 467

Text #15

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.6203. Min Act: 0.0000

Data Index: 1934985 (The Pile)

Max Activating Token Index: 429

Text #16

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.6352. Min Act: 0.0000

Data Index: 4322 (The Pile)

Max Activating Token Index: 287

Text #17

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.5778. Min Act: -0.0000

Data Index: 897617 (The Pile)

Max Activating Token Index: 625

Text #18

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.6266. Min Act: -0.0000

Data Index: 1053244 (The Pile)

Max Activating Token Index: 935

Text #19

Max Range: 0.9730. Min Range: -0.9730

Max Act: 0.6746. Min Act: -0.0000

Data Index: 1451054 (The Pile)

Max Activating Token Index: 615