Reproduce “NavCrafter: Exploring 3D Scenes from a Single Image” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hongbo Duan, Peiyu Zhuang, Yi Liu, Zhengyang Zhang, Yuxin Zhang, Pengting Luo, Fangming Liu, Xueqian Wang
NavCrafter presents a framework for generating controllable, consistent novel views and reconstructing high-fidelity 3D scenes from a single image. The system combines video diffusion models with a multi-stage camera control mechanism, a collision-aware trajectory planner, and an enhanced 3D Gaussian Splatting pipeline, achieving improved visual quality and geometric consistency in novel-view synthesis and 3D reconstruction, with LPIPS of 0.158 and 0.179 respectively on RealEstate10K.
Key insights
- Precise camera trajectory control can be integrated into video diffusion models via a multi-stage architecture involving dual-branch injection and attention modulation, improving novel-view synthesis alignment.
- Iterative scene exploration with a collision-aware trajectory planner enables progressive scene coverage and ensures geometrically valid camera paths, mitigating fragmentation in reconstructed point clouds.
- Enhancements to the 3D Gaussian Splatting pipeline, including depth-aligned supervision, structural regularization, and diffusion-based refinement, are crucial for achieving high-fidelity and geometrically consistent 3D reconstructions from synthesized views.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.