Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3078 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1034. Min Range: -1.1034

Max Act: 1.1034. Min Act: -0.0000

Data Index: 1602062 (The Pile)

Max Activating Token Index: 785

Text #1

Max Range: 1.1034. Min Range: -1.1034

Max Act: 1.0574. Min Act: -0.0000

Data Index: 5717 (The Pile)

Max Activating Token Index: 366

Text #2

Max Range: 1.1034. Min Range: -1.1034

Max Act: 1.0120. Min Act: -0.0000

Data Index: 931627 (The Pile)

Max Activating Token Index: 636

Text #3

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.9593. Min Act: -0.0000

Data Index: 1398052 (The Pile)

Max Activating Token Index: 823

Text #4

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.9610. Min Act: -0.0000

Data Index: 1833626 (The Pile)

Max Activating Token Index: 91

Text #5

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.9090. Min Act: -0.0000

Data Index: 1031480 (The Pile)

Max Activating Token Index: 968

Text #6

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.7145. Min Act: -0.0000

Data Index: 658342 (The Pile)

Max Activating Token Index: 745

Text #7

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.8342. Min Act: -0.0000

Data Index: 806025 (The Pile)

Max Activating Token Index: 830

Text #8

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.8586. Min Act: -0.0000

Data Index: 1428942 (The Pile)

Max Activating Token Index: 383

Text #9

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.7966. Min Act: -0.0000

Data Index: 1534497 (The Pile)

Max Activating Token Index: 667

Text #10

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.8704. Min Act: -0.0000

Data Index: 778705 (The Pile)

Max Activating Token Index: 66

Text #11

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.8163. Min Act: -0.0000

Data Index: 1341217 (The Pile)

Max Activating Token Index: 748

Text #12

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.8364. Min Act: -0.0000

Data Index: 1406916 (The Pile)

Max Activating Token Index: 494

Text #13

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.8535. Min Act: -0.0000

Data Index: 1050346 (The Pile)

Max Activating Token Index: 161

Text #14

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.7903. Min Act: -0.0000

Data Index: 1541138 (The Pile)

Max Activating Token Index: 165

Text #15

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.8251. Min Act: -0.0000

Data Index: 1881703 (The Pile)

Max Activating Token Index: 591

Text #16

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.7302. Min Act: -0.0000

Data Index: 1671082 (The Pile)

Max Activating Token Index: 120

Text #17

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.7673. Min Act: -0.0000

Data Index: 1679308 (The Pile)

Max Activating Token Index: 254

Text #18

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.8150. Min Act: -0.0000

Data Index: 1455100 (The Pile)

Max Activating Token Index: 735

Text #19

Max Range: 1.1034. Min Range: -1.1034

Max Act: 0.7837. Min Act: -0.0000

Data Index: 1489556 (The Pile)

Max Activating Token Index: 83