Reproduce “Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Nathan Kallus, Masatoshi Uehara

The authors develop Double Reinforcement Learning for estimating a target policy’s value from trajectories logged under another policy. It combines a state-action density ratio with a reward prediction and correction, exploiting the Markov decision process structure. The method attains MDP efficiency under stated conditions. In Cliff Walking, its RMSE is 0.075 versus 5.55 for marginalized importance sampling at 1,500 trajectories.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.