Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4305 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2567. Min Act: -0.0000

Data Index: 978955 (The Pile)

Max Activating Token Index: 974

Text #1

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2390. Min Act: -0.0000

Data Index: 1472042 (The Pile)

Max Activating Token Index: 836

Text #2

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2264. Min Act: -0.0000

Data Index: 954555 (The Pile)

Max Activating Token Index: 595

Text #3

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2165. Min Act: -0.0000

Data Index: 1959270 (The Pile)

Max Activating Token Index: 980

Text #4

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2484. Min Act: -0.0000

Data Index: 1886656 (The Pile)

Max Activating Token Index: 803

Text #5

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2543. Min Act: -0.0000

Data Index: 139008 (The Pile)

Max Activating Token Index: 914

Text #6

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2229. Min Act: -0.0000

Data Index: 649007 (The Pile)

Max Activating Token Index: 606

Text #7

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2189. Min Act: -0.0000

Data Index: 15776 (The Pile)

Max Activating Token Index: 463

Text #8

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2157. Min Act: -0.0000

Data Index: 686936 (The Pile)

Max Activating Token Index: 444

Text #9

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2175. Min Act: -0.0000

Data Index: 1861765 (The Pile)

Max Activating Token Index: 325

Text #10

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2366. Min Act: -0.0000

Data Index: 86432 (The Pile)

Max Activating Token Index: 919

Text #11

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2159. Min Act: -0.0000

Data Index: 405436 (The Pile)

Max Activating Token Index: 247

Text #12

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2013. Min Act: -0.0000

Data Index: 803327 (The Pile)

Max Activating Token Index: 982

Text #13

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.1939. Min Act: -0.0000

Data Index: 1341705 (The Pile)

Max Activating Token Index: 1004

Text #14

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2067. Min Act: -0.0000

Data Index: 1442473 (The Pile)

Max Activating Token Index: 277

Text #15

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.1953. Min Act: -0.0000

Data Index: 1609652 (The Pile)

Max Activating Token Index: 925

Text #16

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.2219. Min Act: -0.0000

Data Index: 796392 (The Pile)

Max Activating Token Index: 802

Text #17

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.1971. Min Act: -0.0000

Data Index: 482770 (The Pile)

Max Activating Token Index: 641

Text #18

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.1809. Min Act: -0.0000

Data Index: 1232610 (The Pile)

Max Activating Token Index: 952

Text #19

Max Range: 0.2567. Min Range: -0.2567

Max Act: 0.1897. Min Act: -0.0000

Data Index: 154953 (The Pile)

Max Activating Token Index: 282