Reproduce “Free Geometry: Refining 3D Reconstruction from Longer Versions of Itself” on OpenResearch

About this paper

Yuhang Dai, Xingyi Yang

Free Geometry is a test-time adaptation framework that allows pre-trained feed-forward multi-view 3D reconstruction models to refine their geometric accuracy on novel scenes without requiring 3D ground truth. The method leverages the principle that more input views lead to more reliable reconstructions to create a self-supervision signal for feature-level adaptation, achieving average improvements of 3.73% in camera pose accuracy and 2.88% in point map F1-score across various benchmarks.

Key insights

• The inherent property that more input views generally lead to better 3D reconstruction quality provides a strong, architecturally-guaranteed self-supervision signal for test-time adaptation without requiring explicit ground truth.
• Effective test-time adaptation for multi-view 3D reconstruction should target the feature level within the multi-view transformer backbone, as this is where crucial cross-view reasoning and geometric consistency are established.
• Combining intra-frame feature alignment with cross-frame relational consistency, particularly by preserving feature manifold geometry from masked views, is crucial for robust geometric recalibration and resolving ambiguities under sparse observations.

Summary by alphaXiv.