Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2561 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.2931. Min Act: -0.0000

Data Index: 690904 (The Pile)

Max Activating Token Index: 1015

Text #1

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.2289. Min Act: -0.0000

Data Index: 1885029 (The Pile)

Max Activating Token Index: 779

Text #2

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.2497. Min Act: -0.0000

Data Index: 951136 (The Pile)

Max Activating Token Index: 813

Text #3

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.2276. Min Act: -0.0000

Data Index: 1937299 (The Pile)

Max Activating Token Index: 633

Text #4

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.2226. Min Act: -0.0000

Data Index: 1862308 (The Pile)

Max Activating Token Index: 910

Text #5

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1939. Min Act: -0.0000

Data Index: 1929549 (The Pile)

Max Activating Token Index: 640

Text #6

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1946. Min Act: -0.0000

Data Index: 1234595 (The Pile)

Max Activating Token Index: 946

Text #7

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.2078. Min Act: -0.0000

Data Index: 700593 (The Pile)

Max Activating Token Index: 916

Text #8

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.2076. Min Act: -0.0000

Data Index: 1183158 (The Pile)

Max Activating Token Index: 981

Text #9

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1829. Min Act: -0.0000

Data Index: 50022 (The Pile)

Max Activating Token Index: 592

Text #10

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1897. Min Act: -0.0000

Data Index: 1358272 (The Pile)

Max Activating Token Index: 191

Text #11

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1808. Min Act: -0.0000

Data Index: 1009171 (The Pile)

Max Activating Token Index: 771

Text #12

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1722. Min Act: -0.0000

Data Index: 1441004 (The Pile)

Max Activating Token Index: 29

Text #13

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1810. Min Act: -0.0000

Data Index: 1936116 (The Pile)

Max Activating Token Index: 750

Text #14

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1834. Min Act: -0.0000

Data Index: 669877 (The Pile)

Max Activating Token Index: 165

Text #15

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1693. Min Act: 0.0000

Data Index: 930887 (The Pile)

Max Activating Token Index: 929

Text #16

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1859. Min Act: -0.0000

Data Index: 1343398 (The Pile)

Max Activating Token Index: 58

Text #17

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1750. Min Act: -0.0000

Data Index: 326046 (The Pile)

Max Activating Token Index: 785

Text #18

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1749. Min Act: -0.0000

Data Index: 1438378 (The Pile)

Max Activating Token Index: 1013

Text #19

Max Range: 0.2931. Min Range: -0.2931

Max Act: 0.1839. Min Act: -0.0000

Data Index: 337655 (The Pile)

Max Activating Token Index: 712