Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1424 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.7804. Min Act: -0.0000

Data Index: 1188160 (The Pile)

Max Activating Token Index: 173

Text #1

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.8690. Min Act: -0.0000

Data Index: 644489 (The Pile)

Max Activating Token Index: 573

Text #2

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.6753. Min Act: -0.0000

Data Index: 552489 (The Pile)

Max Activating Token Index: 618

Text #3

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.6664. Min Act: -0.0000

Data Index: 1178732 (The Pile)

Max Activating Token Index: 176

Text #4

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.6741. Min Act: -0.0000

Data Index: 360547 (The Pile)

Max Activating Token Index: 661

Text #5

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.3512. Min Act: -0.0000

Data Index: 1175770 (The Pile)

Max Activating Token Index: 466

Text #6

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.2602. Min Act: -0.0000

Data Index: 960043 (The Pile)

Max Activating Token Index: 505

Text #7

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.3386. Min Act: -0.0000

Data Index: 338748 (The Pile)

Max Activating Token Index: 240

Text #8

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.1969. Min Act: -0.0000

Data Index: 1478046 (The Pile)

Max Activating Token Index: 410

Text #9

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.1395. Min Act: -0.0000

Data Index: 1396793 (The Pile)

Max Activating Token Index: 589

Text #10

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.2979. Min Act: -0.0000

Data Index: 1361323 (The Pile)

Max Activating Token Index: 694

Text #11

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.2765. Min Act: -0.0000

Data Index: 1335891 (The Pile)

Max Activating Token Index: 768

Text #12

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.2155. Min Act: -0.0000

Data Index: 872269 (The Pile)

Max Activating Token Index: 943

Text #13

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.1266. Min Act: -0.0000

Data Index: 885876 (The Pile)

Max Activating Token Index: 236

Text #14

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.2719. Min Act: -0.0000

Data Index: 1408999 (The Pile)

Max Activating Token Index: 133

Text #15

Max Range: 1.8690. Min Range: -1.8690

Max Act: 0.9766. Min Act: -0.0000

Data Index: 1389021 (The Pile)

Max Activating Token Index: 796

Text #16

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.2334. Min Act: -0.0000

Data Index: 1140800 (The Pile)

Max Activating Token Index: 324

Text #17

Max Range: 1.8690. Min Range: -1.8690

Max Act: 0.9596. Min Act: -0.0000

Data Index: 1394524 (The Pile)

Max Activating Token Index: 871

Text #18

Max Range: 1.8690. Min Range: -1.8690

Max Act: 1.1038. Min Act: -0.0000

Data Index: 819062 (The Pile)

Max Activating Token Index: 643

Text #19

Max Range: 1.8690. Min Range: -1.8690

Max Act: 0.6623. Min Act: -0.0000

Data Index: 1089947 (The Pile)

Max Activating Token Index: 876