Reproduce “Mixed Diffusion for 3D Indoor Scene Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siyi Hu, Diego Martin Arroyo, Stephanie Debats, Fabian Manhardt, Luca Carlone, Federico Tombari
Researchers from MIT and Google introduce MiDiffusion, a novel mixed discrete-continuous diffusion model designed for 3D indoor scene synthesis conditioned on floor plans. This model jointly generates object semantic labels and geometric attributes, demonstrating superior realism and geometric plausibility compared to existing methods, and can adapt to various conditional tasks like scene completion without re-training.
Key insights
- Jointly modeling the inherently mixed discrete (semantic) and continuous (geometric) attributes of 3D scenes within a unified diffusion framework leads to substantially more realistic and consistent scene generation.
- Conditioning the denoising process for each data domain on the current latent state of the other domain allows for effective implicit interaction and consistency learning between semantic and geometric scene elements.
- A transformer-based denoising network, combined with an efficient floor plan encoder, is crucial for learning complex inter-object relationships and robustly adhering to spatial boundary constraints.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.