Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2459 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.4908. Min Act: -0.0000

Data Index: 141552 (The Pile)

Max Activating Token Index: 608

Text #1

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.3167. Min Act: -0.0000

Data Index: 837701 (The Pile)

Max Activating Token Index: 206

Text #2

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.1907. Min Act: -0.0000

Data Index: 246610 (The Pile)

Max Activating Token Index: 667

Text #3

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.1484. Min Act: -0.0000

Data Index: 1968043 (The Pile)

Max Activating Token Index: 660

Text #4

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.1572. Min Act: -0.0000

Data Index: 1030475 (The Pile)

Max Activating Token Index: 823

Text #5

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.1377. Min Act: -0.0000

Data Index: 688527 (The Pile)

Max Activating Token Index: 196

Text #6

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.1976. Min Act: -0.0000

Data Index: 301827 (The Pile)

Max Activating Token Index: 744

Text #7

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.1275. Min Act: -0.0000

Data Index: 1576418 (The Pile)

Max Activating Token Index: 444

Text #8

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.0530. Min Act: -0.0000

Data Index: 1408528 (The Pile)

Max Activating Token Index: 984

Text #9

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.0893. Min Act: -0.0000

Data Index: 1587626 (The Pile)

Max Activating Token Index: 408

Text #10

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.0864. Min Act: -0.0000

Data Index: 166126 (The Pile)

Max Activating Token Index: 441

Text #11

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.0794. Min Act: -0.0000

Data Index: 1731619 (The Pile)

Max Activating Token Index: 818

Text #12

Max Range: 1.4908. Min Range: -1.4908

Max Act: 0.8943. Min Act: -0.0000

Data Index: 440309 (The Pile)

Max Activating Token Index: 935

Text #13

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.0195. Min Act: -0.0000

Data Index: 117470 (The Pile)

Max Activating Token Index: 501

Text #14

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.0593. Min Act: -0.0000

Data Index: 1060781 (The Pile)

Max Activating Token Index: 319

Text #15

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.1773. Min Act: -0.0000

Data Index: 37293 (The Pile)

Max Activating Token Index: 253

Text #16

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.0045. Min Act: -0.0000

Data Index: 123289 (The Pile)

Max Activating Token Index: 171

Text #17

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.0174. Min Act: -0.0000

Data Index: 1388233 (The Pile)

Max Activating Token Index: 10

Text #18

Max Range: 1.4908. Min Range: -1.4908

Max Act: 1.0617. Min Act: -0.0000

Data Index: 890597 (The Pile)

Max Activating Token Index: 887

Text #19

Max Range: 1.4908. Min Range: -1.4908

Max Act: 0.9842. Min Act: -0.0000

Data Index: 919153 (The Pile)

Max Activating Token Index: 382