Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 582 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.1342. Min Act: -0.0000

Data Index: 872049 (The Pile)

Max Activating Token Index: 352

Text #1

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.1805. Min Act: -0.0000

Data Index: 518919 (The Pile)

Max Activating Token Index: 23

Text #2

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.9947. Min Act: -0.0000

Data Index: 1016740 (The Pile)

Max Activating Token Index: 230

Text #3

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0523. Min Act: -0.0000

Data Index: 1528641 (The Pile)

Max Activating Token Index: 649

Text #4

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.9886. Min Act: -0.0000

Data Index: 175364 (The Pile)

Max Activating Token Index: 789

Text #5

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0495. Min Act: -0.0000

Data Index: 1301478 (The Pile)

Max Activating Token Index: 102

Text #6

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.9804. Min Act: -0.0000

Data Index: 1959818 (The Pile)

Max Activating Token Index: 578

Text #7

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0051. Min Act: -0.0000

Data Index: 1747217 (The Pile)

Max Activating Token Index: 147

Text #8

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.9814. Min Act: -0.0000

Data Index: 543414 (The Pile)

Max Activating Token Index: 119

Text #9

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.9225. Min Act: -0.0000

Data Index: 612735 (The Pile)

Max Activating Token Index: 261

Text #10

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.9336. Min Act: -0.0000

Data Index: 213926 (The Pile)

Max Activating Token Index: 659

Text #11

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.9616. Min Act: -0.0000

Data Index: 807990 (The Pile)

Max Activating Token Index: 103

Text #12

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.9937. Min Act: -0.0000

Data Index: 239273 (The Pile)

Max Activating Token Index: 148

Text #13

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.9463. Min Act: -0.0000

Data Index: 1459215 (The Pile)

Max Activating Token Index: 117

Text #14

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.8821. Min Act: -0.0000

Data Index: 634058 (The Pile)

Max Activating Token Index: 215

Text #15

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.8501. Min Act: -0.0000

Data Index: 798022 (The Pile)

Max Activating Token Index: 736

Text #16

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.9626. Min Act: -0.0000

Data Index: 436572 (The Pile)

Max Activating Token Index: 673

Text #17

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.9326. Min Act: -0.0000

Data Index: 760000 (The Pile)

Max Activating Token Index: 728

Text #18

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.8805. Min Act: -0.0000

Data Index: 1176575 (The Pile)

Max Activating Token Index: 782

Text #19

Max Range: 1.1805. Min Range: -1.1805

Max Act: 0.8973. Min Act: -0.0000

Data Index: 654272 (The Pile)

Max Activating Token Index: 802