Reproduce “Catch & Carry: Reusable Neural Controllers for Vision-Guided Whole-Body Tasks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Josh Merel, Saran Tunyasuvunakool, Arun Ahuja, Yuval Tassa, Leonard Hasenclever, Vu Pham, Tom Erez, Greg Wayne and 1 more

The authors develop a reusable controller for simulated humanoid object interaction, trained from motion-capture demonstrations and directed by task policies. Its low-level decoder uses proprioception and latent commands, while object state informs training. The shared controller supports box transport and ball catching and tossing from vision or state features, but results remain limited to simulation and depend on training setup.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.