Reproduce “Deep Reinforcement Learning with Gradient Eligibility Traces” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Esraa Elelimy, Brett Daley, Andrew Patterson, Marlos C. Machado, Adam White, Martha White

The authors extend the generalized projected Bellman error objective from one-step to multistep value learning, deriving forward-view sequence-based and backward-view eligibility-trace updates. Gradient PPO improves returns over PPO in Ant and HalfCheetah, with similar performance in Hopper and Walker2d; streaming QRC(λ) outperforms StreamQ and Q(λ) in all five MinAtar games, but establishes no convergence guarantees for nonlinear methods.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.