Reproduce “SPOT: SE(3) Pose Trajectory Diffusion for Object-Centric Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Cheng-Chun Hsu, Bowen Wen, Jie Xu, Yashraj Narang, Xiaolong Wang, Yuke Zhu, Joydeep Biswas, Stan Birchfield
SPOT is an object-centric imitation learning framework that leverages SE(3) object pose trajectories and diffusion models to learn complex robotic manipulation skills. It decouples robot embodiment and automatically learns planning constraints from demonstrations, achieving 79.4% average success on RLBench and successfully transferring skills from just eight iPhone-shot human demonstrations to a robot for tasks like pouring water.
Key insights
- The full SE(3) object pose trajectory, relative to a target object, acts as an effective intermediate representation for implicitly capturing complex task planning constraints from demonstrations.
- Decoupling robot embodiment through object-centric pose trajectories enables learning from diverse, action-less human video demonstrations and facilitates cross-embodiment generalization.
- Employing a diffusion model to synthesize future object pose trajectories provides a robust and generative approach for planning and executing intricate manipulation sequences.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.