Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4094 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.5294. Min Act: -0.0000

Data Index: 497338 (The Pile)

Max Activating Token Index: 179

Text #1

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.5298. Min Act: -0.0000

Data Index: 1657103 (The Pile)

Max Activating Token Index: 864

Text #2

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.5144. Min Act: -0.0000

Data Index: 640404 (The Pile)

Max Activating Token Index: 785

Text #3

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.4954. Min Act: -0.0000

Data Index: 1587528 (The Pile)

Max Activating Token Index: 729

Text #4

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.5568. Min Act: -0.0000

Data Index: 661423 (The Pile)

Max Activating Token Index: 415

Text #5

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.5320. Min Act: -0.0000

Data Index: 1976636 (The Pile)

Max Activating Token Index: 61

Text #6

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.4523. Min Act: -0.0000

Data Index: 1892964 (The Pile)

Max Activating Token Index: 561

Text #7

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.4527. Min Act: -0.0000

Data Index: 1249418 (The Pile)

Max Activating Token Index: 386

Text #8

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.4097. Min Act: -0.0000

Data Index: 1025824 (The Pile)

Max Activating Token Index: 402

Text #9

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.4590. Min Act: -0.0000

Data Index: 683898 (The Pile)

Max Activating Token Index: 752

Text #10

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.3312. Min Act: -0.0000

Data Index: 1743595 (The Pile)

Max Activating Token Index: 818

Text #11

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.3223. Min Act: -0.0000

Data Index: 798359 (The Pile)

Max Activating Token Index: 389

Text #12

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.3026. Min Act: -0.0000

Data Index: 1793400 (The Pile)

Max Activating Token Index: 271

Text #13

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.2839. Min Act: -0.0000

Data Index: 772185 (The Pile)

Max Activating Token Index: 803

Text #14

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.2769. Min Act: -0.0000

Data Index: 91404 (The Pile)

Max Activating Token Index: 308

Text #15

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.2682. Min Act: -0.0000

Data Index: 1637441 (The Pile)

Max Activating Token Index: 86

Text #16

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.1978. Min Act: -0.0000

Data Index: 1060982 (The Pile)

Max Activating Token Index: 870

Text #17

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.2585. Min Act: -0.0000

Data Index: 1604964 (The Pile)

Max Activating Token Index: 726

Text #18

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.3264. Min Act: -0.0000

Data Index: 656548 (The Pile)

Max Activating Token Index: 592

Text #19

Max Range: 1.5568. Min Range: -1.5568

Max Act: 1.2326. Min Act: -0.0000

Data Index: 964248 (The Pile)

Max Activating Token Index: 270