Reproduce “CQF-HMR: Continuous Quaternion Flows for Probabilistic 3D Human Mesh Recovery from a Single Image” on OpenResearch

About this paper

Cuong Le, Bao-Long Tran, Pavlo Melnyk, Tahereh Dehdarirad, Bastian Wandt, Mårten Wadenbäck

Researchers from Linköping University and an independent researcher introduce CQF-HMR, a framework that employs continuous quaternion flows and spherical linear interpolation (slerp) for robustly handling 3D joint rotations within the SMPL model to probabilistically recover 3D human meshes from a single image. This method generates more plausible 3D hypotheses, improving MPJPE by up to 7.6% on the Human3.6M ambiguous split compared to previous probabilistic methods.

Key insights

• Quaternions, with their singularity-free nature and globally defined differential equations, are critical for ensuring smooth and continuous rotational trajectories in generative models when combined with slerp-based optimal transport.
• The flow matching framework provides a stable and efficient learning process for continuous normalizing flows, enabling accurate prediction of angular velocities and shape shifting rates.
• Leveraging sophisticated human body priors (e.g., VPoser) for initial pose sampling significantly enhances the anatomical plausibility and accuracy of generated 3D human meshes, especially in ambiguous scenarios.

Summary by alphaXiv.