Reproduce “Keypose Exploration: Efficient Automatic Trajectory Labelling and Cross-Embodiment Policy Transfer” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yupu Lu, Hang Xu, Yizhou Chen, Jia Pan
A method is introduced for automatically extracting keyposes from robotic demonstrations using vision-language models and trajectory analysis, then leveraging these labels within a Diffusion Policy. This approach, enhanced with a kinematic reachability filter, improves zero-shot cross-embodiment transfer for multimodal tasks, achieving up to an 9.1% increase in success rate on the Square task for a UR5e robot while reducing performance variance.
Key insights
- Automatic keypose extraction can be achieved by combining high-level VLM semantic understanding with low-level trajectory analysis, reducing reliance on manual effort or task-specific heuristics.
- Explicitly conditioning generative policies on keyposes helps to disambiguate multimodal action distributions and enables more precise control during critical task phases.
- Integrating kinematic reachability constraints into keypose selection is a viable strategy for improving zero-shot policy transfer across different robot embodiments by biasing the policy towards feasible actions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.