Reproduce “A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning” on OpenResearch

About this paper

Fahimeh Fakour, Ali Mosleh, Ramin Ramezani

This paper offers a structured review of uncertainty in machine learning and deep learning, presenting a comprehensive framework that classifies uncertainty, identifies its sources, examines quantification techniques, and considers calibration and decision-making processes. It provides a holistic view of the field, highlighting the critical importance of uncertainty quantification for deploying trustworthy ML systems in high-stakes applications.

Key insights

• Uncertainty in machine learning is categorized into aleatoric (inherent randomness, irreducible) and epistemic (lack of knowledge/data, reducible), each requiring different handling strategies.
• Uncertainty arises from diverse sources, including incomplete, noisy, or scarce data, distribution shifts, and model-related factors like architecture limitations or parameter estimation inaccuracies.
• Effective uncertainty quantification requires specific metrics beyond standard performance measures, and critically, the calibration of these uncertainty estimates to ensure predicted probabilities align with empirical frequencies.

Summary by alphaXiv.