Reproduce “DreamMimic: Learning Visuomotor Whole-Body Loco-Manipulation via World Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jie Yin, Xingyu Lai
DreamMimic distills a privileged humanoid manipulation teacher into a vision-based student using a recurrent world model that predicts latent features from visual and proprioceptive history and compares imagined student and teacher actions over three steps. On SMPL-X on OMOMO, it tracks 92.2% of clips, versus 76.5% for Simple-CNN. Evidence is simulation-only, using rendered depth and segmentation, with no real-robot experiments.
Key insights
- Multi-step latent distillation, which aligns future latent dynamics between student and teacher actions, is crucial for maintaining temporally coherent interaction states and mitigating compounding errors under partial observability.
- Providing auxiliary predictions of interaction cues (e.g., object state, contact) to the policy is highly beneficial for acting effectively in contact-rich tasks, demonstrating that inferred privileged information can bridge the observation gap.
- An adaptive distillation schedule like PCG, which adjusts teacher guidance based on performance, is more robust than fixed annealing schedules, preventing premature decay of guidance or over-interference.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.