Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3987 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.5592. Min Act: 0.0000

Data Index: 1779735 (The Pile)

Max Activating Token Index: 654

Text #1

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.6212. Min Act: -0.0000

Data Index: 106144 (The Pile)

Max Activating Token Index: 535

Text #2

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.7502. Min Act: -0.0000

Data Index: 118350 (The Pile)

Max Activating Token Index: 809

Text #3

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.6554. Min Act: -0.0000

Data Index: 1228514 (The Pile)

Max Activating Token Index: 692

Text #4

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.3468. Min Act: -0.0000

Data Index: 1564927 (The Pile)

Max Activating Token Index: 989

Text #5

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.4471. Min Act: -0.0000

Data Index: 631872 (The Pile)

Max Activating Token Index: 550

Text #6

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.4206. Min Act: -0.0000

Data Index: 522027 (The Pile)

Max Activating Token Index: 599

Text #7

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.2520. Min Act: -0.0000

Data Index: 1669682 (The Pile)

Max Activating Token Index: 1007

Text #8

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.2311. Min Act: -0.0000

Data Index: 1503115 (The Pile)

Max Activating Token Index: 916

Text #9

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.1626. Min Act: -0.0000

Data Index: 1415397 (The Pile)

Max Activating Token Index: 570

Text #10

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.2347. Min Act: -0.0000

Data Index: 1090633 (The Pile)

Max Activating Token Index: 261

Text #11

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.3298. Min Act: -0.0000

Data Index: 1480928 (The Pile)

Max Activating Token Index: 559

Text #12

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.2414. Min Act: -0.0000

Data Index: 1714715 (The Pile)

Max Activating Token Index: 451

Text #13

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.2711. Min Act: -0.0000

Data Index: 1262596 (The Pile)

Max Activating Token Index: 922

Text #14

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.2239. Min Act: -0.0000

Data Index: 1686829 (The Pile)

Max Activating Token Index: 911

Text #15

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.1737. Min Act: -0.0000

Data Index: 280514 (The Pile)

Max Activating Token Index: 217

Text #16

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.2498. Min Act: -0.0000

Data Index: 1442328 (The Pile)

Max Activating Token Index: 642

Text #17

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.1409. Min Act: -0.0000

Data Index: 1809022 (The Pile)

Max Activating Token Index: 707

Text #18

Max Range: 1.7502. Min Range: -1.7502

Max Act: 1.1778. Min Act: -0.0000

Data Index: 680637 (The Pile)

Max Activating Token Index: 774

Text #19

Max Range: 1.7502. Min Range: -1.7502

Max Act: 0.9781. Min Act: -0.0000

Data Index: 1999611 (The Pile)

Max Activating Token Index: 166