Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3395 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 4.5629. Min Range: -4.5629

Max Act: 4.5629. Min Act: -0.0000

Data Index: 319526 (The Pile)

Max Activating Token Index: 359

Text #1

Max Range: 4.5629. Min Range: -4.5629

Max Act: 4.1026. Min Act: -0.0000

Data Index: 1981426 (The Pile)

Max Activating Token Index: 657

Text #2

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.8996. Min Act: -0.0000

Data Index: 541426 (The Pile)

Max Activating Token Index: 533

Text #3

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.6886. Min Act: -0.0000

Data Index: 1440527 (The Pile)

Max Activating Token Index: 303

Text #4

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.5862. Min Act: -0.0000

Data Index: 558779 (The Pile)

Max Activating Token Index: 420

Text #5

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.7610. Min Act: -0.0000

Data Index: 394049 (The Pile)

Max Activating Token Index: 999

Text #6

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.3995. Min Act: -0.0000

Data Index: 1059861 (The Pile)

Max Activating Token Index: 115

Text #7

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.5465. Min Act: -0.0000

Data Index: 1117945 (The Pile)

Max Activating Token Index: 437

Text #8

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.4397. Min Act: -0.0000

Data Index: 378045 (The Pile)

Max Activating Token Index: 289

Text #9

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.4300. Min Act: -0.0000

Data Index: 65040 (The Pile)

Max Activating Token Index: 799

Text #10

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.1183. Min Act: -0.0000

Data Index: 679356 (The Pile)

Max Activating Token Index: 908

Text #11

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.5632. Min Act: -0.0000

Data Index: 1677569 (The Pile)

Max Activating Token Index: 205

Text #12

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.3810. Min Act: -0.0000

Data Index: 990212 (The Pile)

Max Activating Token Index: 149

Text #13

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.3394. Min Act: -0.0000

Data Index: 1199170 (The Pile)

Max Activating Token Index: 639

Text #14

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.4546. Min Act: -0.0000

Data Index: 631938 (The Pile)

Max Activating Token Index: 942

Text #15

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.2708. Min Act: -0.0000

Data Index: 956636 (The Pile)

Max Activating Token Index: 483

Text #16

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.2461. Min Act: -0.0000

Data Index: 714329 (The Pile)

Max Activating Token Index: 551

Text #17

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.0854. Min Act: -0.0000

Data Index: 1650740 (The Pile)

Max Activating Token Index: 458

Text #18

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.1951. Min Act: -0.0000

Data Index: 519647 (The Pile)

Max Activating Token Index: 690

Text #19

Max Range: 4.5629. Min Range: -4.5629

Max Act: 3.1317. Min Act: -0.0000

Data Index: 626885 (The Pile)

Max Activating Token Index: 814