Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1381 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.9540. Min Act: -0.0000

Data Index: 1938811 (The Pile)

Max Activating Token Index: 461

Text #1

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.9568. Min Act: -0.0000

Data Index: 464305 (The Pile)

Max Activating Token Index: 447

Text #2

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.8061. Min Act: -0.0000

Data Index: 682407 (The Pile)

Max Activating Token Index: 443

Text #3

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.7273. Min Act: -0.0000

Data Index: 1641661 (The Pile)

Max Activating Token Index: 304

Text #4

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.5508. Min Act: -0.0000

Data Index: 520914 (The Pile)

Max Activating Token Index: 181

Text #5

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.7311. Min Act: -0.0000

Data Index: 412490 (The Pile)

Max Activating Token Index: 542

Text #6

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.5929. Min Act: -0.0000

Data Index: 1160287 (The Pile)

Max Activating Token Index: 763

Text #7

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.4730. Min Act: -0.0000

Data Index: 401660 (The Pile)

Max Activating Token Index: 240

Text #8

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.5304. Min Act: 0.0000

Data Index: 1391590 (The Pile)

Max Activating Token Index: 718

Text #9

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.5357. Min Act: 0.0002

Data Index: 146165 (The Pile)

Max Activating Token Index: 809

Text #10

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.4421. Min Act: 0.0000

Data Index: 949655 (The Pile)

Max Activating Token Index: 922

Text #11

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.5338. Min Act: 0.0000

Data Index: 1251749 (The Pile)

Max Activating Token Index: 988

Text #12

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.5230. Min Act: -0.0000

Data Index: 1353645 (The Pile)

Max Activating Token Index: 128

Text #13

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.5040. Min Act: -0.0000

Data Index: 1193610 (The Pile)

Max Activating Token Index: 450

Text #14

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.5201. Min Act: 0.0002

Data Index: 1765720 (The Pile)

Max Activating Token Index: 903

Text #15

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.4899. Min Act: 0.0001

Data Index: 1708787 (The Pile)

Max Activating Token Index: 20

Text #16

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.4699. Min Act: -0.0000

Data Index: 555129 (The Pile)

Max Activating Token Index: 200

Text #17

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.4623. Min Act: -0.0000

Data Index: 926459 (The Pile)

Max Activating Token Index: 435

Text #18

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.4771. Min Act: 0.0000

Data Index: 345633 (The Pile)

Max Activating Token Index: 499

Text #19

Max Range: 1.9568. Min Range: -1.9568

Max Act: 1.4651. Min Act: -0.0000

Data Index: 420131 (The Pile)

Max Activating Token Index: 132