Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 860 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1767. Min Range: -1.1767

Max Act: 1.1767. Min Act: -0.0000

Data Index: 562064 (The Pile)

Max Activating Token Index: 975

Text #1

Max Range: 1.1767. Min Range: -1.1767

Max Act: 1.0619. Min Act: -0.0000

Data Index: 725886 (The Pile)

Max Activating Token Index: 626

Text #2

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.9473. Min Act: -0.0000

Data Index: 1826493 (The Pile)

Max Activating Token Index: 8

Text #3

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.9741. Min Act: -0.0000

Data Index: 1123356 (The Pile)

Max Activating Token Index: 822

Text #4

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.9532. Min Act: -0.0000

Data Index: 1365304 (The Pile)

Max Activating Token Index: 1003

Text #5

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.9349. Min Act: -0.0000

Data Index: 1293217 (The Pile)

Max Activating Token Index: 458

Text #6

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.8736. Min Act: -0.0000

Data Index: 376742 (The Pile)

Max Activating Token Index: 737

Text #7

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.8898. Min Act: -0.0000

Data Index: 1011844 (The Pile)

Max Activating Token Index: 113

Text #8

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.8597. Min Act: -0.0000

Data Index: 517915 (The Pile)

Max Activating Token Index: 350

Text #9

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.8344. Min Act: -0.0000

Data Index: 1437928 (The Pile)

Max Activating Token Index: 285

Text #10

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.8000. Min Act: -0.0000

Data Index: 322281 (The Pile)

Max Activating Token Index: 37

Text #11

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.8202. Min Act: -0.0000

Data Index: 778789 (The Pile)

Max Activating Token Index: 73

Text #12

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.8160. Min Act: -0.0000

Data Index: 443976 (The Pile)

Max Activating Token Index: 279

Text #13

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.8326. Min Act: -0.0000

Data Index: 1640550 (The Pile)

Max Activating Token Index: 945

Text #14

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.8000. Min Act: -0.0000

Data Index: 1666574 (The Pile)

Max Activating Token Index: 269

Text #15

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7950. Min Act: -0.0000

Data Index: 1693372 (The Pile)

Max Activating Token Index: 8

Text #16

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7352. Min Act: -0.0000

Data Index: 1679340 (The Pile)

Max Activating Token Index: 724

Text #17

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7585. Min Act: -0.0000

Data Index: 1467879 (The Pile)

Max Activating Token Index: 516

Text #18

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7834. Min Act: -0.0000

Data Index: 1210475 (The Pile)

Max Activating Token Index: 548

Text #19

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.8400. Min Act: -0.0000

Data Index: 305273 (The Pile)

Max Activating Token Index: 178