Reproduce “SceneTransporter: Optimal Transport-Guided Compositional Latent Diffusion for Single-Image Structured 3D Scene Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ling Wang, Hao-Xiang Guo, Xinzhou Wang, Fuchun Sun, Kai Sun, Pengkun Liu, Hang Xiao, Zhong Wang and 4 more

SceneTransporter introduces an Optimal Transport-guided framework to generate structured 3D scenes with disentangled, instance-level objects from a single input image. This approach improves geometric fidelity and object coherence, achieving state-of-the-art performance in complex open-world scenarios.

Key insights

  • A 'debiased clustering probe' revealed that while compositional latent spaces implicitly contain object information, models fail to explicitly leverage it for stable instance grouping due to a lack of structural constraints.
  • Optimal Transport provides a principled mechanism to impose explicit structural constraints, enabling clean, capacity-consistent, one-to-one routing of image features to 3D parts, preventing entanglement.
  • Edge-aware affinity smoothing within the OT cost matrix successfully improves disentanglement by preventing information spread across object boundaries.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.