Reproduce “Adaptive Estimator Selection for Off-Policy Evaluation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yi Su, Pavithra Srinath, Akshay Krishnamurthy
SLOPE applies Lepski’s principle to select among off-policy evaluation estimators without observing the target policy’s true value or directly estimating each candidate’s MSE. Its oracle inequality controls the selected estimator by the best available bias–confidence tradeoff, while experiments in continuous-action bandits and reinforcement learning show improved aggregate performance over fixed tuning choices and competitive results against DM, WDR, and MAGIC.
Key insights
- Lepski-style interval intersection avoids estimating the unknown target value or candidate MSE directly; selection relies instead on bias bounds and simultaneous confidence control.
- Under monotonicity conditions, SLOPE achieves an oracle-style error bound within a constant factor of the best candidate bias-plus-confidence tradeoff, including candidates that are biased.
- The selected tuning parameter adapts to data conditions: it favors lower-variance estimators with limited data and shifts toward lower-bias estimators as sample size increases.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.