Reproduce “DreaMo: Articulated 3D Reconstruction From A Single Casual Video” on OpenResearch

About this paper

Tao Tu, Ming-Feng Li, Chieh Hubert Lin, Yen-Chi Cheng, Min Sun, Ming-Hsuan Yang

A template-free framework for articulated 3D shape reconstruction from single casual videos with insufficient view coverage is introduced by DreaMo, combining neural implicit representations with view-conditioned diffusion models. The approach surpasses state-of-the-art methods in visual quality and semantic consistency, yielding improved CLIP scores (e.g., 0.813 vs 0.792 for exhaustive CLIP) and lower Kernel Inception Distance.

Key insights

• The strategic integration of view-conditioned diffusion models with neural implicit representations effectively addresses the challenge of insufficient view coverage in articulated 3D reconstruction.
• Applying Score Distillation Sampling gradients selectively to geometry-relevant parameters (neural bones) is crucial for guiding shape completion without degrading existing texture details or surface fidelity.
• Temporal regularization and cyclic consistency constraints, even for novel views, are essential for learning stable deformations and ensuring plausible articulated models from dynamic, casual video data.

Summary by alphaXiv.