Reproduce “DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yang Zheng, Ruizhi Shao, Yuxiang Zhang, Tao Yu, Zerong Zheng, Qionghai Dai, Yebin Liu
A new method called DeepMultiCap achieves high-fidelity performance capture for multiple characters with sparse multi-view cameras, employing an attention-aware implicit function coupled with parametric body models. It demonstrates superior reconstruction accuracy, particularly reducing Chamfer distance in three-person scenes to 1.130 compared to approximately 1.8 for prior methods.
Key insights
- Self-attention mechanisms can adaptively weigh and aggregate multi-view features more effectively than simple averaging, preserving high-frequency details and handling view inconsistencies.
- Integrating parametric body model priors, specifically through rendered global normal maps, provides valuable guidance to attention networks for robustly reconstructing occluded areas.
- Temporally smoothing Signed Distance Fields (SDFs) of warped meshes across frames can effectively reduce noise and improve consistency in dynamic performance capture.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.