Reproduce “Robotic VLA Benefits from Joint Learning with Motion Image Diffusion” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yu Fang, Kanchana Ranasinghe, Le Xue, Honglu Zhou, Juntao Tan, Ran Xu, Shelby Heinecke, Caiming Xiong and 5 more

Salesforce AI Research and UNC Chapel Hill developed a method to enhance Vision-Language-Action (VLA) models by jointly learning action prediction with optical-flow-based motion image diffusion. This approach improves performance across diverse simulation and real-world robotic manipulation tasks, notably boosting long-horizon task success rates by 4.0% on LIBERO-Long and real-world task success rates by 23%, without increasing inference latency.

Key insights

  • Jointly learning action prediction with explicit, optical-flow-based motion image diffusion forces the VLM backbone to learn temporally coherent and physically grounded representations.
  • Optical-flow-based motion images provide a more dense, control-aligned, and effective supervision signal for dynamics compared to language descriptions or future image predictions.
  • Integrating motion reasoning during training enhances VLA models' temporal understanding and generalization without increasing inference complexity, making it practical for real-time robotic deployment.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.