Reproduce “Sparse Meets Dense: Correspondence Guided Robotic Manipulation with Rigid-Deformable Interactions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ziyu Zhu, Yue Chen, Xirui Liang, Hojin Bae, Yuran Wang, Zhen Yuan, Ruihai Wu, Hao Dong
Researchers at Peking University developed a hybrid correspondence framework that guides robotic manipulation during rigid-deformable object interactions by combining sparse task-specific keypoints with robust dense deformable object tracking. The approach achieved an average success rate of 73.5% in simulation and 65.0% in real-world tasks, outperforming existing methods and demonstrating generalization from a single demonstration.
Key insights
- Combining sparse, task-specific interaction keypoints with robust dense tracking of deformable objects provides a more effective and data-efficient representation for rigid-deformable manipulation.
- Distilling 2D dense correspondence knowledge into a 3D tracking module, then fine-tuning with limited 3D data, significantly improves accuracy and robustness for deformable objects, even under occlusion.
- Framing rigid-deformable interactions as a constraint optimization problem using these hybrid correspondences enables precise and stable robotic actions from a single demonstration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.