Reproduce “Grasp as You Say: Language-guided Dexterous Grasp Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yi-Lin Wei, Jian-Jian Jiang, Chengyi Xing, Xian-Tuo Tan, Xiao-Ming Wu, Hao Li, Mark Cutkosky, Wei-Shi Zheng
Researchers from Sun Yat-sen University and Stanford University developed DexGYSGrasp, a framework enabling natural language-guided dexterous grasping for multi-fingered robots, alongside DexGYSNet, a large-scale, cost-effective dataset for this task. The framework achieves state-of-the-art performance in intention alignment and grasp diversity, with comparable quality, compared to prior methods.
Key insights
- Object penetration loss, while crucial for grasp quality, significantly hinders a single model's ability to learn diverse and intention-aligned dexterous grasp distributions.
- Decomposing the learning process into separate generative (intention/diversity) and regressive (quality refinement) objectives effectively resolves the conflict posed by penetration loss.
- Large-scale, language-guided dexterous grasp datasets can be created cost-effectively by combining human-to-robot grasp retargeting with LLM-assisted natural language annotation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.