Reproduce “Adapt as You Say: Online Interactive Bimanual Skill Adaptation via Human Language Feedback” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhuo Li, Dianxi Li, Tao Teng, Quentin Rouxel, Zhipeng Dong, Dennis Hong, Darwin Caldwell, Fei Chen
Researchers at The Chinese University of Hong Kong and collaborators present BiSAIL, a hierarchical framework for zero-shot online adaptation of bimanual robot skills through human language feedback. It robustly adapts complex dual-arm motions to diverse task variations by decoupling language interpretation from real-time motion modulation.
Key insights
- Decoupling the interpretation of adaptation objectives from motion modulation allows for zero-shot online adaptation and improves interpretability compared to end-to-end learning approaches.
- Integrating Multimodal Large Language Models with embodied augmentation facilitates robust interpretation of ambiguous human language feedback into structured, non-differentiable adaptation objectives and coordination constraints.
- Compositional diffusion sampling, which combines a learned bimanual motion prior with task-specific policies and coordination constraints, effectively generates high-dimensional, coordinated bimanual motions in real-time under implicit language guidance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.