Reproduce “Cost-Sensitive Conformal Training with Provably Controllable Learning Bounds” on OpenResearch

About this paper

Xuesong Jia, Yuanjie Shi, Ziquan Liu, Yi Xu, Yan Yan

Researchers at Washington State University developed Rank-Weighted Cross-Entropy (RWCE) conformal training, a method that directly optimizes prediction set size by minimizing the expected rank of the true label. This approach achieves a 21.38% reduction in average prediction set size on image tasks compared to baselines, while maintaining statistical coverage.

Key insights

• The expected size of any conformal prediction set is provably upper bounded by the expected rank of the true label, plus a small calibration slack term.
• A rank-weighted cross-entropy objective can serve as an upper bound for the expected rank, enabling its minimization through standard gradient descent without needing to approximate the non-differentiable indicator function for prediction set size.
• Treating the true-label rank as an importance weight in the cross-entropy loss provides a cost-sensitive signal that effectively drives the model to improve the ranking of true labels, directly leading to smaller prediction sets.

Summary by alphaXiv.