Reproduce “Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kristopher De Asis, Alan Chan, Silviu Pitis, Richard S. Sutton, Daniel Graves
The authors introduce fixed-horizon temporal-difference learning, which predicts rewards over successive horizons, each bootstrapping from a shorter forecast rather than itself. This structure avoids feedback between horizons; one-step learning uses importance-sampling corrections, while shared weights support a deep control version. It converges on Baird’s off-policy counterexample and outperforms tested DQN settings on LunarLander-v2 without target networks, results limited to setups.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.