Reproduce “Replicable Conformal Prediction” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Marios Papamichalis, Regina Ruane, Theofanis Papamichalis

This research introduces RECAL (REplicable CALibration), a procedure for conformal prediction that produces identical set-valued classifiers across independent calibrations by rounding thresholds to a shared-seed random grid. The method preserves marginal coverage guarantees while providing ρ-replicability and robustness against selective recalibration attacks, with quantified trade-offs in sample size and prediction set inflation.

Key insights

  • Exact replication of conformal predictors is impossible without shared randomness, and a data-dependent, exactly replicable procedure cannot simultaneously be uniformly valid.
  • Achieving ρ-replicability (agreement within a small probability ρ) while maintaining coverage guarantees introduces unavoidable costs in terms of increased sample size and prediction set inflation, for which tight lower bounds are established.
  • The shared-seed RECAL procedure robustly protects against selective recalibration attacks by bounding the probability of an adversary selecting a distinct, potentially undercovering classifier.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.