Reproduce “DreamMesh: Jointly Manipulating and Texturing Triangle Meshes for Text-to-3D Generation” on OpenResearch

About this paper

Haibo Yang, Yang Chen, Yingwei Pan, Ting Yao, Zhineng Chen, Zuxuan Wu, Yu-Gang Jiang, Tao Mei

DreamMesh presents a framework for text-to-3D generation that directly produces high-fidelity explicit triangle meshes with corresponding textures from natural language prompts. The method achieves superior quantitative and qualitative results on the T3Bench benchmark, yielding organized geometry and realistic textures directly usable in standard 3D pipelines.

Key insights

• Directly optimizing explicit triangle meshes from the start effectively overcomes the inherent limitations of implicit and hybrid 3D representations, leading to cleaner geometry and more practical assets.
• A coarse-to-fine scheme, progressively refining both global structure and fine details, is crucial for robust and high-quality joint mesh deformation and texturing from text prompts.
• Leveraging 2D diffusion models for normal map guidance in the coarse stage and a diffusion-based image refiner for joint texture-geometry refinement in the fine stage is effective for achieving high fidelity.

Summary by alphaXiv.