Reproduce “Bi-KVIL: Keypoints-based Visual Imitation Learning of Bimanual Manipulation Tasks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jianfeng Gao, Xiaoshu Jin, Franziska Krebs, Noémie Jaquier, Tamim Asfour
Bi-KVIL enables robots to learn complex bimanual manipulation tasks from a small number of human visual demonstrations by automatically extracting explicit hierarchical object-hand relationships and keypoints-based geometric constraints. The system demonstrates robust generalization to novel objects and scene configurations, reproducing various manipulation styles on a humanoid robot.
Key insights
- Explicitly modeling Hybrid Master-Slave Relationships (HMSR) and bimanual coordination strategies enables data-efficient learning and robust generalization for bimanual tasks.
- Combining symbolic HMSR with sub-symbolic keypoints-based geometric constraints allows the system to consistently identify task structures while capturing fine-grained manipulation styles.
- A sophisticated perception pipeline leveraging off-the-shelf vision models can reliably extract 3D object and hand trajectories from RGB-D cameras for complex bimanual imitation learning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.