Reproduce “Geometry-guided Cross-view Diffusion for One-to-many Cross-view Image Synthesis” on OpenResearch

About this paper

Tao Jun Lin, Wenqing Wang, Yujiao Shi, Akhil Perincherry, Ankit Vora, Hongdong Li

Researchers from The Australian National University and collaborators introduce Geometry-guided Cross-view Diffusion, a framework that generates diverse and geometrically consistent target-view images (e.g., ground to satellite or vice-versa) from a single input view. This approach leverages diffusion models with explicit 3D geometric guidance to capture the inherent one-to-many relationship in cross-view image synthesis, outperforming prior methods in fidelity and diversity.

Key insights

• Explicit 3D geometric guidance, when integrated into a generative model, is crucial for resolving pose ambiguity and ensuring structural consistency in cross-view image synthesis, especially in scenarios with varying camera orientations.
• Diffusion models effectively address the one-to-many nature of cross-view synthesis by generating diverse images that reflect variations in illumination, weather, and dynamic objects, a limitation of prior deterministic methods.
• Projecting multi-level features instead of raw pixels or simple embeddings enhances robustness against geometric distortions and improves the model's ability to establish dense spatial correspondence between views.

Summary by alphaXiv.