Reproduce “Conformal Off-Policy Prediction in Contextual Bandits” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Muhammad Faaiz Taufiq, Jean-Francois Ton, Rob Cornish, Yee Whye Teh, Arnaud Doucet
COPP uses logged outcomes from a behavior policy to predict context-specific sets of plausible outcomes under a target policy. It reweights calibration scores to account for policy shift, rather than reporting an average. In a synthetic experiment at nominal 90% coverage, COPP with estimated weights achieved 0.91 versus 0.87 for direct model sampling. This is marginal evidence from one setting.
Key insights
- Framing the policy shift as a change in P(Y|X) allows for the application of weighted Conformal Prediction to generate valid predictive intervals for off-policy outcomes.
- Covariate-adaptive predictive intervals can be achieved by incorporating a weight function that depends on both covariates (X) and outcomes (Y), making the uncertainty quantification context-specific.
- The COPP framework extends to stochastic target policies and continuous action spaces, broadening the applicability of Conformal Prediction in complex decision-making scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.