Reproduce “MoCaP: Mobile Code-as-Policies for Zero-Shot Mobile Manipulation from Passive Human Videos” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yiyang Cai, Tatsuya Kamijo, Ninghao Lu, Zijin Hu, Sherry Yang

MoCaP uses passive human video to help a mobile robot find objects and approach them from reachable positions. It builds scene memory for search and estimates a person’s pre-reach standing direction as an approach cue, while robot-grounded planning checks executable motions. In tests, it completed 16/20 box and 8/10 can grasp-and-hold trials; tested CaP-X baselines completed none, in one environment.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.