Reproduce “Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Process” on OpenResearch

About this paper

Jiayi Chen, Wenxuan Song, Pengxiang Ding, Ziyang Zhou, Han Zhao, Feilong Tang, Donglin Wang, Haoang Li

UD-VLA introduces a Vision-Language-Action (VLA) model that synchronously generates future images and robot actions using a Joint Discrete Denoising Diffusion Process (JD3P) within a single transformer. This approach achieves state-of-the-art performance on multiple robot manipulation benchmarks, including an average success rate of 92.7% on LIBERO, and demonstrates robust real-world generalization with 4.3 times faster inference than autoregressive methods.

Key insights

• Tightly coupling future image generation and action prediction via a joint discrete diffusion process intrinsically strengthens their synergy, allowing actions to be iteratively refined under continuous and evolving visual guidance.
• A hybrid attention mechanism, balancing intra-modal bidirectional interactions with causal cross-modal information flow, is crucial for effectively integrating multimodal tokens without leakage.
• Explicitly modeling and utilizing future visual information as a 'chain-of-thought' through multi-step diffusion provides richer temporal cues, significantly benefiting robot action planning and improving generalization capabilities.

Summary by alphaXiv.