Reproduce “PROGRESSOR: A Perceptually Guided Reward Estimator with Self-Supervised Online Refinement” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tewodros Ayalew, Xiao Zhang, Kevin Yuanbo Wu, Tianchong Jiang, Michael Maire, Matthew R. Walter
PROGRESSOR presents a method for learning robot reward functions directly from unlabeled video demonstrations by estimating task progress, enabling robots to acquire complex behaviors without explicit supervision. The approach combines self-supervised progress prediction with adversarial online refinement to overcome challenges of reward engineering and data labeling, achieving improved performance on simulated and real-world manipulation tasks.
Key insights
- Expert video demonstrations inherently contain temporal information about task progress, which can be leveraged for self-supervised reward learning.
- Treating reward estimation as a progress prediction problem allows for dense reward signals without manual engineering or action labels.
- An adversarial 'push-back' mechanism during online reinforcement learning effectively mitigates distribution shift between expert demonstrations and agent exploration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.