Reproduce “DefGoalNet: Contextual Goal Learning from Demonstrations For Deformable Object Manipulation” on OpenResearch

About this paper

Bao Thach, Tanner Watts, Shing-Hei Ho, Tucker Hermans, Alan Kuntz

DefGoalNet, developed by researchers at the University of Utah and NVIDIA, introduces a neural network that autonomously learns to predict desired deformable object goal shapes from a small number of human demonstrations, overcoming a critical bottleneck in shape servoing methods. This approach enables shape servoing to be more practical for real-world robotic tasks, demonstrated by achieving nearly 90% success in surgical retraction with only 10 simulated demonstrations and successful zero-shot sim-to-real transfer.

Key insights

• Decomposing deformable object manipulation into contextual goal learning and goal-conditioned shape control allows for modularity and data-efficient training.
• Learning goal shapes from a small number of demonstrations (as few as 10-100) significantly enhances the practical applicability of shape servoing for real-world robotic tasks.
• The ability to visualize predicted goal shapes as an intermediate step offers increased transparency and safety, particularly crucial in sensitive applications like surgery.

Summary by alphaXiv.