Reproduce “Learning Granularity-Aware Affordances from Human-Object Interaction for Tool-Based Functional Dexterous Grasping” on OpenResearch

About this paper

Fan Yang, Wenrui Chen, Kailun Yang, Haoran Lin, Dongsheng Luo, Conghui Tang, Zhiyong Li, Yaonan Wang

Researchers at Hunan University developed GAAF-Dex, a weakly supervised framework that learns from human-object interactions to enable robots to perform functional dexterous grasping. The system precisely localizes task-specific functional regions and predicts coarse multi-finger gestures, achieving improvements in affordance grounding with a 5.1% lower KLD and 16.3% higher NSS over prior methods, alongside an average gesture prediction accuracy of 91.3%.

Key insights

• Weakly supervised learning from human-object interactions can effectively guide both fine-grained functional region localization and coarse gesture prediction, significantly reducing annotation efforts.
• Leveraging multi-granular affordance features (precise finger contact and overall grasp type) is essential for enabling versatile and functional tool use with dexterous robots.
• A novel dataset (FAH) with real paired exocentric and egocentric views, task-specific functional annotations, and robot-agnostic coarse gesture labels provides a critical benchmark for advancing functional manipulation research.

Summary by alphaXiv.