Reproduce “DM0: An Embodied-Native Vision-Language-Action Model towards Physical AI” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
En Yu, Haoran Lv, Jianjian Sun, Kangheng Lin, Ruitao Zhang, Yukang Shi, Yuyang Chen, Ze Chen and 41 more
Dexmal & StepFun introduces DM0, an "Embodied-Native" Vision-Language-Action (VLA) model, which unifies manipulation and navigation for generalist robots by integrating physical understanding from initial training stages. The model achieved a 62.0% average success rate on the RoboChallenge Table30 tasks in a specialist setting and demonstrated superior generalization compared to existing VLA models.
Key insights
- Integrating physical priors from the onset of training, rather than adapting purely semantic models, is more effective for developing robustly physically grounded generalist robot policies.
- A hybrid gradient strategy, which decouples action expert learning from the Vision-Language Model's (VLM) semantic preservation, effectively prevents catastrophic forgetting and maintains broad reasoning capabilities while acquiring precise motor control.
- Embodied Spatial Scaffolding, through hierarchical supervision from subtask decomposition to precise trajectory prediction, allows for learning a coherent understanding that bridges high-level semantics with low-level continuous actions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.