Reproduce “Guided Co-Modulated GAN for 360° Field of View Extrapolation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mohammad Reza Karimi Dastjerdi, Yannick Hold-Geoffroy, Jonathan Eisenmann, Siavash Khodadadeh, Jean-François Lalonde
ImmerseGAN, developed by researchers at Université Laval and Adobe, introduces a guided co-modulated GAN for extrapolating full 360-degree panoramas from single limited-FOV images. It enables user control over the semantic content of extrapolated regions, achieving improved visual quality and seamless panoramas, and facilitates realistic virtual object insertion.
Key insights
- Integrating FOV-specific training masks and a horizontal shifting operation within a GAN generator effectively enables the production of high-quality, seamless, wrap-around 360-degree panoramas from limited-view images.
- A pre-trained discriminative network can be effectively repurposed to provide intuitive semantic guidance for generative models, allowing users to direct the content of extrapolated regions without complex latent space inversions.
- High-fidelity 360-degree environment generation, coupled with semantic control, is crucial for enhancing realism in applications such as virtual object compositing, especially for objects with reflective surfaces.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.