Reproduce “Rank2Reward: Learning Shaped Reward Functions from Passive Video” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Daniel Yang, Davin Tjia, Jacob Berg, Dima Damen, Pulkit Agrawal, Abhishek Gupta
Rank2Reward demonstrates learning effective reward functions for robotic manipulation directly from passive video demonstrations. It leverages the temporal ordering of video frames to infer task progress, enabling robots to learn complex behaviors without manual reward engineering.
Key insights
- Temporal ordering of frames in expert video demonstrations provides rich information about task progression.
- Learning a utility function that ranks states based on their temporal order allows for the construction of a dense, well-shaped reward function.
- Combining this learned ranking-based reward with adversarial imitation learning guides robot exploration effectively while staying close to expert behavior.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.