Reproduce “Orthogonal Witness Control for Muon Optimization via Sigmoid Spectral Reshaping” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dat Phi Van, Ngo Vu Minh, Tuc Nguyen, Thin Nguyen, Ngoc-Thanh Dinh, Trung Le
Soren reshapes matrix-update singular values with a bounded, increasing sigmoid, preserving their ordering while compressing differences rather than flattening them as Muon does. A finite-depth Soft Newton–Schulz implementation avoids explicit singular value decomposition. In supervised fine-tuning on Qwen2.5-3B, Soren averages 64.82 across six benchmarks versus HTMuon’s 63.33; this does not establish that reshaping caused the difference or that results generalize.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.