Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3740 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.2445. Min Act: -0.0000

Data Index: 1963008 (The Pile)

Max Activating Token Index: 840

Text #1

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.2381. Min Act: -0.0000

Data Index: 985227 (The Pile)

Max Activating Token Index: 840

Text #2

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.2157. Min Act: -0.0000

Data Index: 1227524 (The Pile)

Max Activating Token Index: 943

Text #3

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.2074. Min Act: -0.0000

Data Index: 1592105 (The Pile)

Max Activating Token Index: 816

Text #4

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.2047. Min Act: -0.0000

Data Index: 559156 (The Pile)

Max Activating Token Index: 554

Text #5

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.1749. Min Act: -0.0000

Data Index: 376601 (The Pile)

Max Activating Token Index: 755

Text #6

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.2029. Min Act: -0.0000

Data Index: 1349170 (The Pile)

Max Activating Token Index: 848

Text #7

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.2194. Min Act: -0.0000

Data Index: 331809 (The Pile)

Max Activating Token Index: 715

Text #8

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.1997. Min Act: -0.0000

Data Index: 722759 (The Pile)

Max Activating Token Index: 976

Text #9

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.2121. Min Act: -0.0000

Data Index: 1856093 (The Pile)

Max Activating Token Index: 993

Text #10

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.2012. Min Act: -0.0000

Data Index: 1200707 (The Pile)

Max Activating Token Index: 1012

Text #11

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.2076. Min Act: -0.0000

Data Index: 1188543 (The Pile)

Max Activating Token Index: 971

Text #12

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.1836. Min Act: -0.0000

Data Index: 564357 (The Pile)

Max Activating Token Index: 750

Text #13

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.1935. Min Act: -0.0000

Data Index: 1597155 (The Pile)

Max Activating Token Index: 884

Text #14

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.1802. Min Act: -0.0000

Data Index: 749646 (The Pile)

Max Activating Token Index: 873

Text #15

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.2009. Min Act: -0.0000

Data Index: 237522 (The Pile)

Max Activating Token Index: 970

Text #16

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.1927. Min Act: -0.0000

Data Index: 963343 (The Pile)

Max Activating Token Index: 807

Text #17

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.2112. Min Act: -0.0000

Data Index: 593780 (The Pile)

Max Activating Token Index: 933

Text #18

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.1918. Min Act: -0.0000

Data Index: 1765898 (The Pile)

Max Activating Token Index: 853

Text #19

Max Range: 0.2445. Min Range: -0.2445

Max Act: 0.1905. Min Act: -0.0000

Data Index: 800079 (The Pile)

Max Activating Token Index: 967