Reproduce “Diagnosing Conformal Prediction Failures Under Distribution Shift: A COVID-19 Case Study” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chorok Lee

The authors evaluate SHAP concentration as a pre-deployment warning for conformal prediction coverage loss under data shift. It measures the leading feature’s share of total mean absolute attribution on validation data. Across 16 multiclass tasks, concentration correlated with coverage drop (Spearman rho 0.853); the association is strongest for LightGBM and is not a universal guarantee.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.