Reproduce “Deep Visual Foresight for Planning Robot Motion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chelsea Finn, Sergey Levine
Deep Visual Foresight turns action-conditioned video prediction into a robot controller. It predicts pixel motion from images, actions, and robot state, scores candidate pushes against user-selected pixel goals, executes one action, and replans after each observation. Across ten real-robot pushing tasks, it measured 2.52 ± 1.06 pixels versus 3.19 ± 1.68 for a replanning end-effector-vector heuristic, limited evidence for short-horizon pushing.
Key insights
- Deep video prediction models, trained purely on unlabeled interaction videos, can implicitly learn effective physical dynamics for robotic control.
- High-level manipulation goals can be translated into actionable robot commands solely by specifying desired pixel movements, bypassing explicit object models or semantic parsing.
- An end-to-end learned approach significantly reduces the need for explicit calibration, instrumentation, and detailed physical models, enhancing robustness to novel objects and environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.