Reproduce “VectorAdam for Rotation Equivariant Geometry Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Selena Ling, Nicholas Sharp, Alec Jacobson
VectorAdam replaces Adam’s coordinate-wise second moments with one scalar per vector gradient norm, preserving update direction while adapting step size. The authors prove rotation equivariance for rotation-invariant objectives. In a 200-vertex mesh test, VectorAdam and gradient descent show uniform direction distributions, unlike Adam. Across eight Nefertiti reconstructions, VectorAdam has lower Hausdorff distance; this does not establish that it prevents all self-intersections.
Key insights
- Equivariance is a critical consideration not just for model architectures and loss functions, but also for the underlying optimization process, especially when dealing with structured vector data like geometric coordinates.
- Adam's per-scalar gradient statistics fundamentally distort the direction of vector-valued gradients in early optimization steps, leading to biased updates towards coordinate axes.
- A conceptually simple modification to an optimizer's second moment calculation can restore crucial geometric symmetries, improving optimization stability and quality without compromising convergence rates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.