Reproduce “Doubly Robust Policy Evaluation and Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Miroslav Dudik, John Langford, Lihong Li
A doubly robust (DR) technique was developed for offline policy evaluation and optimization in contextual bandit settings, achieving unbiased policy value estimates if either the reward function or the historical logging policy is correctly modeled. This approach demonstrated up to a 13.6% reduction in RMSE for policy evaluation and consistently led to lower classification errors in policy optimization tasks compared to existing methods.
Key insights
- Theoretical analysis confirms the DR estimator's unbiasedness if either the reward model or the historical propensity score model is perfectly specified.
- Non-asymptotic bias analysis shows the DR estimator's bias is proportional to the product of the reward model's and propensity model's deviations from truth.
- Variance analysis indicates that DR can reduce variance compared to IPS, especially when a reasonable reward estimator is available, offering a better bias-variance trade-off.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.