Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1745 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.7967. Min Range: -1.7967

Max Act: 1.7967. Min Act: -0.0000

Data Index: 300990 (The Pile)

Max Activating Token Index: 437

Text #1

Max Range: 1.7967. Min Range: -1.7967

Max Act: 1.3732. Min Act: -0.0000

Data Index: 1054502 (The Pile)

Max Activating Token Index: 462

Text #2

Max Range: 1.7967. Min Range: -1.7967

Max Act: 1.2141. Min Act: -0.0000

Data Index: 836591 (The Pile)

Max Activating Token Index: 470

Text #3

Max Range: 1.7967. Min Range: -1.7967

Max Act: 1.1435. Min Act: -0.0000

Data Index: 1576985 (The Pile)

Max Activating Token Index: 273

Text #4

Max Range: 1.7967. Min Range: -1.7967

Max Act: 1.1766. Min Act: -0.0000

Data Index: 106210 (The Pile)

Max Activating Token Index: 618

Text #5

Max Range: 1.7967. Min Range: -1.7967

Max Act: 1.0847. Min Act: -0.0000

Data Index: 96464 (The Pile)

Max Activating Token Index: 386

Text #6

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.9393. Min Act: -0.0000

Data Index: 19529 (The Pile)

Max Activating Token Index: 906

Text #7

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.9375. Min Act: -0.0000

Data Index: 102969 (The Pile)

Max Activating Token Index: 18

Text #8

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.9949. Min Act: -0.0000

Data Index: 1401493 (The Pile)

Max Activating Token Index: 432

Text #9

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.9688. Min Act: -0.0000

Data Index: 1409971 (The Pile)

Max Activating Token Index: 669

Text #10

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.9685. Min Act: -0.0000

Data Index: 1146768 (The Pile)

Max Activating Token Index: 527

Text #11

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.8938. Min Act: -0.0000

Data Index: 429789 (The Pile)

Max Activating Token Index: 377

Text #12

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.9757. Min Act: -0.0000

Data Index: 1140531 (The Pile)

Max Activating Token Index: 106

Text #13

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.9529. Min Act: -0.0000

Data Index: 351218 (The Pile)

Max Activating Token Index: 7

Text #14

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.8483. Min Act: -0.0000

Data Index: 1663229 (The Pile)

Max Activating Token Index: 806

Text #15

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.8934. Min Act: -0.0000

Data Index: 1174394 (The Pile)

Max Activating Token Index: 137

Text #16

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.8409. Min Act: -0.0000

Data Index: 140654 (The Pile)

Max Activating Token Index: 344

Text #17

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.9795. Min Act: -0.0000

Data Index: 349747 (The Pile)

Max Activating Token Index: 212

Text #18

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.8810. Min Act: -0.0000

Data Index: 1116547 (The Pile)

Max Activating Token Index: 482

Text #19

Max Range: 1.7967. Min Range: -1.7967

Max Act: 0.8462. Min Act: -0.0000

Data Index: 893634 (The Pile)

Max Activating Token Index: 250