Reproduce “Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D priors” on OpenResearch

About this paper

Lihe Ding, Shaocong Dong, Zhanpeng Huang, Zibin Wang, Yiyuan Zhang, Kaixiong Gong, Dan Xu, Tianfan Xue

A Bidirectional Diffusion (BiDiff) framework unifies 2D and 3D diffusion processes through mutual guidance to generate 3D objects from text, producing results in approximately 40 seconds with a CLIP R-Precision of 0.79. This approach enables independent control over object geometry and texture, and significantly reduces artifacts like multi-face issues, making 3D content creation more efficient.

Key insights

• Mutual regularization through bidirectional guidance forces synchronous evolution of 2D and 3D diffusion processes, creating a unified generative prior that leverages both types of knowledge.
• The framework enables independent control over geometry and texture generation by allowing users to adjust separate guidance scales during inference, offering enhanced creative flexibility.
• Effectively combining the strengths of 2D and 3D priors addresses long-standing trade-offs, leading to both high-quality texture and geometric consistency in generated 3D objects.

Summary by alphaXiv.