Reproduce “Generalization and Informativeness of Weighted Conformal Risk Control Under Covariate Shift” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Matteo Zecchin, Fredrik Hellström, Sangwoo Park, Shlomo Shamai, Osvaldo Simeone
W-CRC extends conformal risk control to covariate shift and bounds expected prediction-set size under assumptions, linking it to predictor generalization, shift, and data allocation. In Sigfox localization experiments with 1,500 examples, fixed-budget splits performed worse at both training extremes, and satisfactory splits favored more calibration as shift increased. The result guides allocation under stated conditions.
Key insights
- The severity of covariate shift (quantified by the maximum likelihood ratio) directly increases W-CRC's inefficiency by affecting base predictor generalization and the bound's decay rate.
- Increasing the calibration set size significantly improves W-CRC efficiency by enabling more precise threshold estimation and mitigating adverse effects of covariate shift.
- Optimal data splitting between training and calibration sets is dynamic: severe covariate shift necessitates allocating a larger proportion of data to calibration for better informativeness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.