Reproduce “Dream2Reward: Transition-Alignment Reward Models from Positive Demonstrations for Robotic Manipulation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haoyu Zhang, Zecui Zeng, Bin Wang, Lusong Li, Liang Lin, Long Cheng

Dream2Reward learns a language-conditioned latent transition field from positive robot demonstrations, providing dense, causal rewards that evaluate observed robot motion by aligning its latent displacement with a predicted successful change. This approach achieved an ROC-AUC of 0.95 for failure detection and boosted real-robot offline reinforcement learning success rates to 34.5% on contact-rich tasks.

Key insights

  • Evaluating robot behavior based on transition-level alignment, considering both direction and magnitude of latent displacement, provides a more robust and granular reward signal than state-level progress.
  • A causal reward, derived by comparing observed motion with a predicted successful local change, can effectively penalize common robotic failures (wrong direction, overshooting, stalling) using only positive demonstrations.
  • The proposed reward formulation is theoretically stable under bounded perturbations and can resolve ambiguity where different behaviors lead to states with similar scalar progress values.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.