Reproduce “Sharpness-Aware Muon: Matrix-Aware Optimization with Momentum-Guided Perturbations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhiwei Zhai, Wenjing Yan, Ying-Jun Angela Zhang
MoMuSAM extends Muon with a local loss probe, using accumulated matrix momentum to perturb weights before computing an update. It uses one gradient evaluation per iteration, compared with MuSAM’s two. On CIFAR-100 with ResNet-18, MoMuSAM reports 75.14±0.16% test accuracy, versus 74.62±0.17% for MuSAM and 74.18±0.20% for Muon; this comparison does not establish the cause of the improvement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.