Reproduce “Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation” on OpenResearch

About this paper

Kuan Fang, Yuke Zhu, Animesh Garg, Silvio Savarese, Li Fei-Fei

The CAVIN Planner introduces a model-based hierarchical planning algorithm for multi-step robotic manipulation, employing cascaded variational inference to learn latent representations of high-level effects and low-level motions. It demonstrates superior performance and robustness in complex, long-horizon tasks across varying scene complexities and reward structures, with successful zero-shot transfer from simulation to real-world environments.

Key insights

• Multi-step manipulation tasks exhibit an inherent hierarchical structure that can be exploited by decoupling high-level desired effects (subgoals) from low-level motion generation.
• Learning and planning within distinct latent spaces for effects and motions allows for efficient pruning of unfavorable high-level trajectories, thereby reducing the search space for long-horizon problems.
• An end-to-end cascaded variational inference framework enables joint learning of these hierarchical latent dynamics solely from task-agnostic, self-supervised data.

Summary by alphaXiv.