Reproduce “GeoNVS: Geometry Grounded Video Diffusion for Novel View Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Minjun Kang, Inkyu Shin, Taeyeop Lee, Myungchul Kim, In So Kweon, Kuk-Jin Yoon
GeoNVS guides diffusion denoising by rendering features from a 3D Gaussian scene representation rather than supplying rendered RGB images. It adaptively fuses these geometry-conditioned features with the generator’s internal features. In a DL3DV trajectory test with six reference views, GeoNVS reports translation, rotation, and Chamfer Distance errors of 13.27 cm, 2.01°, and 0.41, lower than SEVA and the tested input-level geometry injection. This result is limited to that setting and baseline.
Key insights
- Integrating explicit 3D geometry priors into the feature space of video diffusion models, rather than at the input level, significantly enhances geometric consistency and camera controllability without introducing artifacts.
- An adaptive fusion strategy, which predicts confidence in geometric priors, is essential for robustly leveraging structural guidance and mitigating sensitivity to prior quality.
- The modular GS-Adapter design allows for zero-shot compatibility with diverse feed-forward 3D geometry models and diffusion backbones, enabling broad applicability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.