Reproduce “Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nathan Kallus, Masatoshi Uehara
The authors develop Double Reinforcement Learning for estimating a target policy’s value from trajectories logged under another policy. It combines a state-action density ratio with a reward prediction and correction, exploiting the Markov decision process structure. The method attains MDP efficiency under stated conditions. In Cliff Walking, its RMSE is 0.075 versus 5.55 for marginalized importance sampling at 1,500 trajectories.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.