Reproduce “$π^3$: Permutation-Equivariant Visual Geometry Learning” on OpenResearch

About this paper

Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang and 2 more

π³ removes the need for a designated reference image in multi-view 3D reconstruction. It predicts each view’s point map, camera pose, and confidence map in its own coordinates, using permutation-equivariant processing. When frame order changes, its mean Accuracy standard deviation is 0.003 on DTU and 0.000 on ETH3D, versus VGGT’s 0.033 and 0.049. This tests ordering robustness, not noisy inputs.

Key insights

• The dependency on a fixed reference view in visual geometry reconstruction constitutes an unnecessary inductive bias that limits the performance and robustness of existing models.
• A fully permutation-equivariant architecture, which treats all input views equally, can fundamentally enhance the stability and consistency of 3D reconstructions.
• By predicting scale-invariant local point maps and affine-invariant relative camera poses, the system can achieve high accuracy and superior robustness to input variations without relying on a global coordinate system defined by a single view.

Summary by alphaXiv.