Reproduce “MatMart: Material Reconstruction of 3D Objects via Diffusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiuchao Wu, Pengfei Zhu, Jiangjing Lyu, Xinguo Liu, Jie Guo, Yanwen Guo, Weiwei Xu, Chengfei Lyu
MatMart introduces a diffusion-based framework for high-fidelity 3D object material reconstruction from RGB images, preserving fine details and ensuring multi-view consistency across arbitrary input counts. The framework achieves superior performance, including an SSIM of 0.945, PSNR of 32.10, FID of 26.20, and LPIPS of 0.052 on rendered images compared to existing methods.
Key insights
- Decoupling material estimation for observed regions from prior-guided generation for unobserved regions within a unified framework improves overall fidelity.
- The novel View-Material Cross-Attention (VMCA) mechanism allows for efficient, scalable multi-view consistency enforcement without the quadratic memory costs of traditional cross-view attention.
- Generating materials in view space, guided by geometric and previously estimated material priors, produces higher quality and more semantically coherent results than direct UV-space generation for unknown regions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.