Reproduce “EasyMimic: A Low-Cost Framework for Robot Imitation Learning from Human Videos” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tao Zhang, Song Xia, Ye Wang, Qin Jin

Researchers at the AIM3 Lab at Renmin University of China developed EasyMimic, a low-cost framework for robots to learn manipulation skills from human video demonstrations. The framework uses physical and visual alignment modules along with a co-training strategy to achieve an average success rate of 0.88 across various tasks, notably improving over robot-only baselines (0.26-0.51) with a hardware cost under $300.

Key insights

  • Effective robot learning from human videos for low-cost platforms requires both kinematic and visual alignment to bridge the embodiment gap.
  • A co-training approach combining abundant human demonstrations with a small amount of real robot data is superior and more efficient for adapting VLA models than sequential methods.
  • Independent action heads for human and robot data within a shared VLA architecture are crucial for stable and high-performing policy learning.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.