Reproduce “Joint Flow Trajectory Optimization For Feasible Robot Motion Generation from Video Demonstrations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaoxiang Dong, Matthew Johnson-Roberson, Weiming Zhi
Researchers from Vanderbilt University, Carnegie Mellon University, and The University of Sydney developed the Joint Flow Trajectory Optimization (JFTO) framework for robot motion generation from video demonstrations. The framework jointly optimizes initial robot grasp poses and subsequent full trajectories, ensuring kinematic feasibility and collision avoidance while precisely imitating object-centric guides from multi-modal human actions using flow matching.
Key insights
- Jointly optimizing the grasp pose and the entire subsequent trajectory is crucial for generating robust and faithful robot motions, outperforming sequential approaches that often lead to suboptimal or infeasible solutions.
- Flow matching effectively addresses multi-modal demonstration distributions by guiding the robot to reproduce specific, observed human strategies rather than collapsing to an unrealistic or physically impossible average.
- Focusing on abstract, object-centric guides extracted from human videos, instead of direct hand imitation, successfully bridges the embodiment gap between human demonstrators and robot manipulators.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.