Reproduce “Riemannian Stochastic Optimization for Sufficient Dimension Reduction” on OpenResearch

About this paper

Thibault Pautrel, François Portier

Researchers from CentraleSupélec, Université Paris-Saclay, and ENSAI, CREST developed SMAVE (Stochastic MAVE), a Riemannian stochastic optimization algorithm for sufficient dimension reduction. SMAVE achieves 10-35 times faster runtimes and up to 9 times lower subspace estimation error compared to prior MAVE methods on synthetic data, while maintaining competitive predictive accuracy on real-world datasets in high-dimensional settings.

Key insights

• The population MAVE objective formally targets the same Grassmannian as OPG, aiming to recover the principal eigenspace of the regression function's gradient outer product matrix.
• Formulating the MAVE objective on the Stiefel manifold allows for a closed-form Riemannian gradient that intrinsically lies in the tangent space, simplifying optimization on this manifold.
• Integrating sparse k-NN localization in the projected space with Riemannian stochastic gradient ascent significantly enhances the scalability of gradient-based SDR methods without compromising statistical performance.

Summary by alphaXiv.