Reproduce “What VGGT Knows About Overlap: Probing Geometric Foundation Models for Co-Visibility” on OpenResearch

About this paper

Filippo Ziliotto, Luciano Serafini, Lamberto Ballan, Tommaso Campari

Co-VGGT is a lightweight model leveraging a frozen Visual Geometry Grounded Transformer (VGGT) backbone to accurately predict image co-visibility, even in low-overlap and sparse-scene conditions. This approach surpasses existing methods and human baselines on the Co-VisiON benchmark, providing well-calibrated predictions useful for 3D reconstruction pipelines.

Key insights

• VGGT's internal representations inherently encode meaningful co-visibility signals, demonstrated by a zero-shot baseline achieving competitive performance without any training.
• Co-visibility reasoning within VGGT exhibits hierarchical layer specialization, with late layers (L12-L24) predominantly contributing to co-visibility judgments, while early layers encode more general geometric primitives.
• The MoE gating mechanism provides interpretability, identifying specific layers (e.g., L17 as a 'negative anchor') specialized in confidently rejecting non-co-visible pairs or contributing to positive overlap detection.

Summary by alphaXiv.