Reproduce “MMaDA: Multimodal Large Diffusion Language Models” on OpenResearch

About this paper

Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, Mengdi Wang

MMaDA introduces a unified diffusion architecture that simultaneously handles textual reasoning, multimodal understanding, and text-to-image generation through a modality-agnostic design. It leverages novel post-training methods, including Mixed Long Chain-of-Thought fine-tuning and a Unified Reinforcement Learning algorithm (UniGRPO), to achieve competitive performance across these diverse tasks, notably improving reasoning and factual consistency.

Key insights

• A truly unified diffusion architecture can effectively serve as a general-purpose foundation model for multimodal intelligence, challenging the prevailing reliance on autoregressive or hybrid approaches.
• Sophisticated post-training strategies, such as Cold Start Long-CoT fine-tuning and a diffusion-adapted RL algorithm (UniGRPO), are critical for boosting reasoning, factual consistency, and overall performance in non-autoregressive multimodal models.
• Diffusion models' inherent parallelism and masked token prediction objective provide significant efficiency advantages and architectural flexibility, naturally supporting tasks like inpainting and extrapolation across modalities.

Summary by alphaXiv.