Reproduce “RoboEdit: Turning Human Manipulation Videos into Scalable Robot Experience” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yaowei Guo, Zeng Tao, Yuxin Jiang, Yunuo Chen, Zhiyang Dou, Yuxiang Ma, Yin Yang, Demetri Terzopoulos and 2 more

RoboEdit turns RGB human manipulation videos into videos of a chosen robot and estimates its hand’s 3D trajectory. Its pipeline automatically creates paired human-robot training clips, while a video editor uses sparse robot keyframes and a state decoder. On a 300-case benchmark, it reaches SSIM 0.9282, versus 0.8996 for VACE; this result is limited to that benchmark protocol.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.