Reproduce “Render the Robot, Generate the World: One Video World Model Across Embodiments” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yash Jangir, Karan Mirakhor, Renos Zabounidis, Muhammad Zubair Irshad, Sergey Zakharov, Yonatan Bisk, Homanga Bharadhwaj

Doppel predicts future video from an initial image and planned robot actions by rendering the robot’s camera-aligned poses and conditioning a shared video model on both renderings and numerical actions. Across three robot-arm datasets, Doppel-8 improves whole-frame PSNR over action-only experts by 1.00 dB on held-out trajectories and 1.20 dB on OOD tests; evaluation covers embodiments seen during training.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.