Reproduce “Fast Escape, Slow Convergence: Learning Dynamics of Phase Retrieval under Power-Law Data” on OpenResearch

About this paper

Guillaume Braun, Bruno Loureiro, Ha Quang Minh, Masaaki Imaizumi

This research rigorously characterizes learning dynamics in phase retrieval, a nonlinear regression problem, under anisotropic Gaussian inputs with power-law spectral decay. It demonstrates a three-phase learning trajectory, providing explicit scaling laws that quantify how the input spectrum dictates the Mean-Squared Error decay, notably showing an "escape-convergence trade-off."

Key insights

• Strong data anisotropy qualitatively reshapes nonlinear learning dynamics, leading to an infinite hierarchy of coupled ODEs and an "escape-convergence trade-off" where initial escape might be fast but subsequent MSE reduction is slow.
• The learning trajectory robustly follows three phases: an initial rapid escape from mediocrity, followed by slower convergence of macroscopic statistics, and finally, a prolonged phase of learning directions associated with small eigenvalues.
• The spectral exponent 'a' directly dictates the pace of learning; a larger 'a' (faster spectral decay) leads to a faster initial escape and faster convergence of summary statistics, but the overall MSE decay rate is still profoundly influenced by the spectral tail.

Summary by alphaXiv.