Reproduce “Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lukas Höllein, Ang Cao, Andrew Owens, Justin Johnson, Matthias Nießner
Text2Room, developed by researchers from Technical University of Munich and the University of Michigan, introduces a method to create textured 3D meshes of room-scale indoor scenes from text prompts, effectively leveraging 2D text-to-image diffusion models. The approach generates dense, coherent, and explorable 3D environments, outperforming baselines in quantitative metrics such as CLIP Score (28.02) and user-rated 3D Structure Completeness (4.19).
Key insights
- Large-scale 2D text-to-image models can be effectively leveraged for complex 3D scene generation by iteratively lifting 2D views into a coherent 3D structure.
- Consistent depth alignment and geometric filtering are crucial for integrating noisy monocular depth predictions from multiple views into a seamless 3D mesh.
- A guided, multi-stage viewpoint selection strategy is essential for progressively building complete, watertight room-scale environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.