Reproduce “Spectrally-Corrected and Regularized Linear Discriminant Analysis for Spiked Covariance Model” on OpenResearch

About this paper

Hua Li, Wenya Luo, Zhidong Bai, Huanchao Zhou, Zhangni Pu

Spectrally-Corrected and Regularized LDA (SRLDA) is introduced as an enhanced classification method for high-dimensional data under a spiked covariance model, offering analytically derived optimal regularization parameters and an optimal intercept. The method consistently outperforms traditional LDA and shows competitive accuracy against other classifiers while effectively performing supervised dimensionality reduction on datasets like MNIST and LFW.

Key insights

• Rigorous asymptotic analysis using large-dimensional random matrix theory allows for the derivation of globally optimal and consistent estimators for regularization and intercept parameters in high-dimensional classification contexts.
• Leveraging the inherent structure of the spiked covariance model, particularly by spectrally correcting eigenvalues and regularizing their impact, significantly improves LDA's robustness and accuracy in high dimensions.
• Small spiked eigenvalues, often disregarded in other studies, play an important role in improving algorithm performance, especially when they are 'hidden in the noise,' and their inclusion is beneficial for accurate modeling.

Summary by alphaXiv.