Reproduce “WQLCP: Weighted Adaptive Conformal Prediction for Robust Uncertainty Quantification Under Distribution Shifts” on OpenResearch

About this paper

Shadi Alijani, Homayoun Najjaran

WQLCP (Weighted Adaptive Conformal Prediction) offers a framework for robust uncertainty quantification in deep learning models that encounter distribution shifts. It utilizes Variational Autoencoder (VAE) reconstruction loss to dynamically adapt Conformal Prediction, which leads to improved coverage guarantees and smaller prediction set sizes across various types of data shifts.

Key insights

• VAE reconstruction loss effectively quantifies the severity of distribution shifts and epistemic uncertainty, providing a robust signal for adapting Conformal Prediction.
• Dynamically adjusting the Conformal Prediction calibration quantile by weighting calibration samples based on their alignment with test-specific uncertainties (as in WQLCP) achieves a superior trade-off between coverage and prediction set size.
• Vision Transformer (ViT) architectures, when used as backbones for VAEs, consistently demonstrate greater robustness and consistency in uncertainty quantification under distribution shifts compared to CNN-based VAEs.

Summary by alphaXiv.