Reproduce “Progress Reward Modeling for Robotic Learning: A Comprehensive Survey” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jianshu Zhang, Keliang Wu, Haoran Lu, Anbang Liu, Ce Zhang, Weijie Yin, Chengxuan Qian, Xiyuan Yang and 3 more

The survey maps progress rewards for long-horizon robot tasks, distinguishing what models observe, how they specify goals, produce signals, obtain supervision, and are evaluated. It separates fidelity, robustness, and downstream utility, showing that scalar calibration, temporal consistency, and policy improvement answer different questions. It reports no unified numerical comparison proving progress rewards generally improve robots.

Key insights

  • The field employs a wide spectrum of approaches for representing task states, specifying goals, and outputting progress signals, reflecting the multifaceted nature of 'progress' researchers aim to capture.
  • Different construction mechanisms for progress rewards rely on distinct underlying assumptions about the source and nature of the progress signal.
  • Significant heterogeneity exists in data generation strategies (human-driven to automated) and evaluation benchmarks, implying that reported 'success' often pertains to specific properties rather than a holistic measure.
  • Several fundamental trade-offs are identified, such as balancing input context richness with online usability and human semantic accuracy with automated scalability.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.