Reproduce “Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexity” on OpenResearch

About this paper

Mouïn Ben Ammar, David Brellmann, Arturo Mendoza, Antoine Manzanera, Gianni Franchi

An investigation into how model complexity affects out-of-distribution (OOD) detection performance demonstrates that the double descent phenomenon, previously observed in generalization, also manifests in post-hoc OOD detection. The study shows that while OOD performance follows a double descent curve, overparameterization does not uniformly improve OOD detection across all architectures, with benefits correlating to the quality of learned representations as quantified by Neural Collapse metrics.

Key insights

• The double descent phenomenon extends beyond in-distribution generalization to post-hoc OOD detection, demonstrating a similar U-shaped risk curve as model complexity increases.
• Overparameterization does not universally guarantee superior OOD detection; its effectiveness depends on the architecture's ability to form structured, separable latent representations, as explained by Neural Collapse (NC) metrics.
• The presence of label noise is crucial for clearly observing the double descent effect in OOD detection, suggesting that model robustness to noisy data is intertwined with this phenomenon.

Summary by alphaXiv.