Reproduce “UniGeo: Unifying Geometric Guidance for Camera-Controllable Image Editing via Video Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hong Jiang, Wensong Song, Zongxin Yang, Ruijie Quan, Yi Yang
UniGeo is a framework for camera-controllable image editing that synthesizes novel views of a scene while strictly preserving cross-view geometric consistency by integrating unified geometric guidance across representation, architecture, and loss function levels of a video diffusion model. This approach achieved superior visual quality and geometric fidelity compared to existing methods across multiple public benchmarks, notably reducing LPIPS from 0.3008 to 0.2377 on RealEstate10K for extensive motions.
Key insights
- Robust cross-view geometric consistency in camera-controllable image editing is best achieved by systematically integrating geometric guidance across all three fundamental levels of a generative model: representation, architecture, and loss function.
- Leveraging the inherent continuous-viewpoint modeling capabilities of video diffusion models, augmented with comprehensive geometric enforcement, can overcome the limitations of discrete view mappings in image diffusion-based approaches.
- Frame-decoupled injection of 3D geometry and an architectural attention mechanism anchored to the initial view effectively stabilize structural coherence throughout a generated video sequence, mitigating geometric drift and degradation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.