Reproduce “Doubly-Robust Estimation of Counterfactual Policy Mean Embeddings” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Houssam Zenati, Bariscan Bozkurt, Arthur Gretton
The paper develops distributional off-policy evaluation for stochastic policies by representing the policy-induced outcome distribution with a kernel mean embedding and estimating it with a doubly robust correction that combines an outcome model and logging-policy propensities. In dSprites experiments, DR-KPT-RBF rejected 2.4% under the null and 65.6% under policy shift, but identification requires consistency, exchangeability, and positivity.
Key insights
- Kernel Mean Embeddings offer a flexible, nonparametric way to represent and compare full outcome distributions, including for complex, high-dimensional, or structured data.
- Doubly robust estimation significantly enhances the reliability and efficiency of distributional OPE by providing robustness against nuisance parameter model misspecification and achieving faster convergence rates.
- The framework allows not only for robust distributional evaluation and comparison but also for practical sampling from estimated counterfactual distributions, useful for policy simulation and data generation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.