Reproduce “DrS: Learning Reusable Dense Rewards for Multi-Stage Tasks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tongzhou Mu, Minghua Liu, Hao Su
DrS, a method developed by researchers at UC San Diego, learns reusable dense reward functions for multi-stage reinforcement learning tasks by training discriminators to differentiate between successful and failed trajectories within specific task stages. This approach achieves comparable or superior performance to complex human-engineered rewards on unseen task variants, significantly improving sample efficiency and success rates in robotic manipulation.
Key insights
- Training a discriminator to distinguish between success and failure trajectories provides a consistent and reusable reward signal, even at convergence, unlike traditional adversarial imitation learning discriminators.
- Decomposing tasks into multiple stages provides richer supervisory signals for reward learning, enabling more effective guidance compared to single-stage or sparse rewards.
- Learned reward functions can offer a more flexible and transferable form of task knowledge across varying environmental parameters than learned policies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.