Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3513 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1517. Min Act: -0.0000

Data Index: 1952883 (The Pile)

Max Activating Token Index: 313

Text #1

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1532. Min Act: -0.0000

Data Index: 1147453 (The Pile)

Max Activating Token Index: 336

Text #2

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.2067. Min Act: -0.0000

Data Index: 1414433 (The Pile)

Max Activating Token Index: 902

Text #3

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1085. Min Act: -0.0000

Data Index: 1708132 (The Pile)

Max Activating Token Index: 688

Text #4

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.9507. Min Act: -0.0000

Data Index: 543061 (The Pile)

Max Activating Token Index: 62

Text #5

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.8662. Min Act: -0.0000

Data Index: 926878 (The Pile)

Max Activating Token Index: 126

Text #6

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.9304. Min Act: -0.0000

Data Index: 137311 (The Pile)

Max Activating Token Index: 897

Text #7

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.0332. Min Act: -0.0000

Data Index: 239753 (The Pile)

Max Activating Token Index: 1006

Text #8

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.8226. Min Act: -0.0000

Data Index: 1673103 (The Pile)

Max Activating Token Index: 951

Text #9

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.9141. Min Act: -0.0000

Data Index: 143559 (The Pile)

Max Activating Token Index: 1012

Text #10

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.8582. Min Act: -0.0000

Data Index: 1156176 (The Pile)

Max Activating Token Index: 743

Text #11

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.7557. Min Act: -0.0000

Data Index: 1390563 (The Pile)

Max Activating Token Index: 785

Text #12

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.7758. Min Act: -0.0000

Data Index: 1505371 (The Pile)

Max Activating Token Index: 471

Text #13

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.7736. Min Act: -0.0000

Data Index: 1519238 (The Pile)

Max Activating Token Index: 964

Text #14

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.8127. Min Act: -0.0000

Data Index: 643762 (The Pile)

Max Activating Token Index: 76

Text #15

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.6959. Min Act: -0.0000

Data Index: 714348 (The Pile)

Max Activating Token Index: 305

Text #16

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.6925. Min Act: -0.0000

Data Index: 577234 (The Pile)

Max Activating Token Index: 746

Text #17

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.7055. Min Act: -0.0000

Data Index: 1175236 (The Pile)

Max Activating Token Index: 843

Text #18

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.6586. Min Act: -0.0000

Data Index: 765393 (The Pile)

Max Activating Token Index: 411

Text #19

Max Range: 1.2067. Min Range: -1.2067

Max Act: 0.7052. Min Act: -0.0000

Data Index: 190848 (The Pile)

Max Activating Token Index: 310