Reproduce “Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinyao Qin, Junjie Lu, Kaixin Wang, Chuheng Zhang, Sinjae Kang, Kimin Lee, Min Xu, Bin Liang and 2 more
A framework called ReTVL enables robots to learn mistake-sensitive value functions from imperfect human demonstrations by leveraging sparsely annotated retry events. It captures local value drops and rebounds during errors and corrections, resulting in more robust robot policies with an 80% success rate on real-world manipulation tasks.
Key insights
- Retry events, traditionally discarded as noise, contain valuable information for understanding execution errors and recovery, offering a low-cost annotation opportunity.
- Explicitly learning local value dynamics, such as temporary value drops during mistakes and rebounds during corrections, significantly improves the accuracy of value functions.
- Combining global absolute progress supervision with local retry-induced pairwise preferences allows for the creation of value functions that are both globally coherent and locally mistake-sensitive.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.