Reproduce “Multi-Keypoint Affordance Representation for Functional Dexterous Grasping” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fan Yang, Dongsheng Luo, Wenrui Chen, Jiacheng Lin, Junjie Cai, Kailun Yang, Zhiyong Li, Yaonan Wang
This work introduces a multi-keypoint affordance representation for functional dexterous grasping, enabling robots to precisely localize task-relevant object interaction points and translate them into executable hand poses. The system achieved a 40 percentage point improvement in Functional Grasp Success compared to prior state-of-the-art on real-world tasks and significantly improved affordance grounding metrics on the FAH dataset.
Key insights
- Integrating LVMs with a hierarchical feature fusion and a two-step clustering process effectively extracts fine-grained, structurally coherent candidate keypoints for dexterous tasks.
- Weakly-supervised learning, by aligning ego-view keypoint features with exo-view object-part prototypes, successfully transfers human contact geometry knowledge without direct keypoint annotations.
- Defining hand and object coordinate systems based on functionally relevant keypoints allows for robust and direct computation of the hand-object relative pose, bridging perception to action.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.