Reproduce “K-VIL: Keypoints-based Visual Imitation Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jianfeng Gao, Zhi Tao, Noémie Jaquier, Tamim Asfour
K-VIL enables robots to acquire manipulation skills from few human visual demonstrations by automatically extracting sparse, object-centric geometric constraints and employing a prioritized admittance controller. The approach achieves high success rates and sub-millimeter accuracy across various tasks, demonstrating robust generalization to novel object instances and large shape variations using as few as 3-4 demonstrations.
Key insights
- Sparse, object-centric keypoints, rather than dense pixel-level data, are effective for building viewpoint-invariant and embodiment-independent task representations.
- Explicitly defining and learning geometric constraints (point-to-point, point-to-line, point-to-plane, point-to-curve, point-to-surface) from demonstration variability enables robust generalization across categorical object variations.
- Combining these geometric constraints with Movement Primitives (VMPs) and a prioritized admittance controller allows for flexible and robust task reproduction, even when extrapolating to novel object shapes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.