Reproduce “Language-Guided Object-Centric Diffusion Policy for Generalizable and Collision-Aware Robotic Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hang Li, Qian Feng, Zhi Zheng, Jianxiang Feng, Zhaopeng Chen, Alois Knoll
Lan-o3dp introduces a robotic manipulation framework integrating object-centric 3D representations, natural language guidance via LLMs, and a novel cost-guided diffusion policy for generalizable and collision-aware task execution. The framework achieved an average success rate of 68.7% across 21 RLBench tasks, surpassing prior diffusion-based methods, and enabled effective zero-shot collision avoidance against novel obstacles in real-world robot experiments.
Key insights
- Object-centric 3D representations, derived from open-vocabulary segmentation, provide more robust and generalizable visual observations for manipulation policies compared to full scene or 2D inputs.
- Leveraging Large Language Models (LLMs) allows for natural language-guided open-vocabulary manipulation and zero-shot collision avoidance with user-defined obstacles.
- Applying cost-based guidance to the *estimated clean trajectory* within a diffusion model's denoising process is crucial for effective and training-free collision avoidance, unlike guiding noisy intermediate steps.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.