Reproduce “Conformal Risk Training: End-to-End Optimization of Conformal Risk Control” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Christopher Yeh, Nicolas Christianson, Adam Wierman, Yisong Yue

The authors extend CRC to OCE risks, including CVaR, and introduce CRT, which differentiates through risk control during model fine-tuning. In polyp segmentation at target FNR 0.05, CRT reports FNR 0.049 and FPR 0.385, versus 0.048 and 0.524 for post-hoc CRC. Guarantees are marginal and depend on the paper’s stated assumptions.

Key insights

  • Conformal Risk Control can be theoretically extended to control a wide class of Optimized Certainty-Equivalent (OCE) risks, including CVaR, by transforming loss functions while maintaining crucial monotonicity properties.
  • Integrating risk control procedures directly into the model training process through differentiable optimization allows models to learn 'risk-aware' representations, leading to better average-case performance without compromising provable guarantees.
  • The ability to compute gradients through the solution of the inner risk control problem, essential for end-to-end optimization, is feasible for scenarios with piecewise constant or convex loss and risk functions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.