Reproduce “MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wei Cheng, Juncheng Mu, Xianfang Zeng, Xin Chen, Anqi Pang, Chi Zhang, Zhibin Wang, Bin Fu and 3 more
MVPaint, developed by Tencent PCG and collaborators, introduces a three-stage coarse-to-fine framework for generating high-resolution, multi-view consistent 3D textures from text, robustly handling complex meshes. The framework achieves top performance on the Objaverse T2T benchmark, with FID 20.89 and KID 3.45, alongside superior human evaluation for overall quality and seamlessness.
Key insights
- Generating multi-view consistent images from the outset, using a Text-to-Multi-View diffusion model and image-domain synchronization, is crucial for avoiding inconsistencies like the Janus problem.
- Addressing untextured regions and seams in 3D space, rather than solely 2D UV space, allows for robustness against fragmented UV unwrapping and improves overall texture continuity.
- A coarse-to-fine strategy combining low-resolution synchronized generation with higher-resolution refinement and 3D-aware post-processing effectively balances consistency, detail, and quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.