Reproduce “Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera” on OpenResearch

About this paper

Hongrui Cai, Wanquan Feng, Xuetao Feng, Yan Wang, Juyong Zhang

Researchers from the University of Science and Technology of China and Alibaba Group developed Neural-DynamicReconstruction (NDR), a framework that reconstructs high-fidelity geometry and motion of dynamic scenes from monocular RGB-D camera input. This system employs neural implicit functions and a novel invertible deforming network, achieving lower geometry error than prior fusion-based methods and effectively modeling topology changes.

Key insights

• Strict cycle consistency, enforced through a neural invertible deforming network, is essential for robustly modeling non-rigid motion and preventing error accumulation over dynamic sequences.
• The joint optimization of neural implicit SDF and radiance fields, fully leveraging both RGB and depth information, effectively resolves monocular ambiguities and enhances both geometric fidelity and textural richness.
• Augmenting the canonical 3D representation with topology-aware coordinates enables neural implicit functions to accurately model scenes undergoing significant topology changes, such as opening mouths or wrinkling skin.

Summary by alphaXiv.