Reproduce “Modiff: Action-Conditioned 3D Motion Generation with Denoising Diffusion Probabilistic Models” on OpenResearch

About this paper

Mengyi Zhao, Mengyuan Liu, Bin Ren, Shuling Dai, Nicu Sebe

Modiff, an action-conditioned 3D motion generation framework from Beihang University, Sun Yat-sen University, University of Pisa, and University of Trento, applies Denoising Diffusion Probabilistic Models (DDPMs) to synthesize realistic and diverse skeleton-based human motions. It achieved a Fréchet Motion Distance (FMD) of 9.12 on the NTU RGB+D dataset, outperforming the previous state-of-the-art Kinetic-GAN (FMD 9.99), while also demonstrating improved diversity and multimodality.

Key insights

• DDPMs are highly effective for action-conditioned 3D human motion generation, producing more realistic and diverse motions than leading GAN-based methods.
• Representing 3D skeleton sequences as 2D image matrices is a practical approach that enables the successful application of established image-based diffusion architectures.
• Architectural and training choices, such as a high-capacity U-Net, L1 loss function, and effective label embedding, are critical for optimal performance in diffusion-based motion synthesis.

Summary by alphaXiv.