Reproduce “DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physics” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sizhe Li, Zhiao Huang, Tao Chen, Tao Du, Hao Su, Joshua B. Tenenbaum, Chuang Gan
DexDeform is a framework that learns dexterous manipulation of deformable objects using multi-fingered robot hands by combining skill abstraction from human demonstrations with differentiable physics for trajectory refinement. The approach achieves a normalized improvement in Earth Mover's Distance of up to 0.972 across various complex manipulation tasks, outperforming existing reinforcement learning and trajectory optimization methods.
Key insights
- Intelligent initializations derived from human-demonstrated skill abstractions significantly mitigate the local optima problem in differentiable physics-based trajectory optimization for deformable objects.
- Learning a latent skill space from demonstrations allows for efficient long-horizon planning by abstracting away low-level finger movements and compositionally understanding deformation.
- A bootstrapping loop that uses differentiable physics to explore novel goals and refine trajectories enables the system to generalize beyond initial demonstrations and continuously improve data efficiency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.