Reproduce “On the Practicality of Deterministic Epistemic Uncertainty” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Janis Postels, Mattia Segu, Tao Sun, Luca Sieber, Luc Van Gool, Fisher Yu, Federico Tombari
An evaluation of Deterministic Uncertainty Methods (DUMs) demonstrates that while they are computationally efficient and effective for out-of-distribution detection, most exhibit poor calibration under continuous distributional shifts compared to established Bayesian methods. Only the Spectral-normalized Gaussian Process (SNGP) consistently showed better calibration, and DUMs scaled successfully to semantic segmentation tasks while retaining their calibration deficiencies.
Key insights
- DUMs that rely on generative models of hidden representations to quantify uncertainty generally exhibit worse calibration, indicating that feature locations alone may not be sufficient for reliable prediction correctness judgments.
- There is a limited correlation between bi-Lipschitz regularization strength and calibration quality for several DUMs, suggesting that simply increasing "distance awareness" does not reliably improve uncertainty estimates.
- SNGP, which derives uncertainty from a predictive distribution using Gaussian Processes, consistently demonstrated better calibration among DUMs, highlighting the importance of integrating the predictive model into uncertainty quantification.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.