Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2062 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0852. Min Range: -1.0852

Max Act: 1.0852. Min Act: -0.0000

Data Index: 912845 (The Pile)

Max Activating Token Index: 956

Text #1

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.9457. Min Act: -0.0000

Data Index: 104978 (The Pile)

Max Activating Token Index: 787

Text #2

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.8505. Min Act: -0.0000

Data Index: 253417 (The Pile)

Max Activating Token Index: 405

Text #3

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.8521. Min Act: -0.0000

Data Index: 592233 (The Pile)

Max Activating Token Index: 1004

Text #4

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.7981. Min Act: -0.0000

Data Index: 1113154 (The Pile)

Max Activating Token Index: 308

Text #5

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.7512. Min Act: -0.0000

Data Index: 1690080 (The Pile)

Max Activating Token Index: 377

Text #6

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.7601. Min Act: -0.0000

Data Index: 1633525 (The Pile)

Max Activating Token Index: 952

Text #7

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.6873. Min Act: -0.0000

Data Index: 1261875 (The Pile)

Max Activating Token Index: 750

Text #8

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.7083. Min Act: -0.0000

Data Index: 794603 (The Pile)

Max Activating Token Index: 468

Text #9

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.7400. Min Act: -0.0000

Data Index: 111870 (The Pile)

Max Activating Token Index: 877

Text #10

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.6694. Min Act: -0.0000

Data Index: 1167655 (The Pile)

Max Activating Token Index: 962

Text #11

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.7239. Min Act: -0.0000

Data Index: 1912844 (The Pile)

Max Activating Token Index: 873

Text #12

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.6716. Min Act: -0.0000

Data Index: 1019456 (The Pile)

Max Activating Token Index: 714

Text #13

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.6476. Min Act: -0.0000

Data Index: 916945 (The Pile)

Max Activating Token Index: 739

Text #14

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.6476. Min Act: -0.0000

Data Index: 1926643 (The Pile)

Max Activating Token Index: 601

Text #15

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.6683. Min Act: -0.0000

Data Index: 1644707 (The Pile)

Max Activating Token Index: 59

Text #16

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.6437. Min Act: -0.0000

Data Index: 1739281 (The Pile)

Max Activating Token Index: 862

Text #17

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.6434. Min Act: -0.0000

Data Index: 1178312 (The Pile)

Max Activating Token Index: 704

Text #18

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.6398. Min Act: -0.0000

Data Index: 985901 (The Pile)

Max Activating Token Index: 759

Text #19

Max Range: 1.0852. Min Range: -1.0852

Max Act: 0.6101. Min Act: -0.0000

Data Index: 418748 (The Pile)

Max Activating Token Index: 203