Reproduce “MeshFM: 2D Features Are All You Need for 3D Shape Understanding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jinfan Zhou, Richard Liu, Itai Lang, Rana Hanocka
MeshFM predicts reusable 3D features in one forward pass, using a point-cloud network trained on teacher fields distilled from 2D vision-model features. Its features support segmentation, correspondence, and deformation without task-specific feature fine-tuning. On PartObjaverse-Tiny semantic part segmentation, it reaches 0.549 mean intersection-over-union versus PartField’s 0.542, with clustering informed by ground-truth label counts.
Key insights
- Appropriately distilled and refined 2D features from foundation models are sufficient for comprehensive, general-purpose 3D shape understanding, negating the need for complex 3D-specific priors or extensive 3D annotations.
- Addressing feature aliasing and bleeding through segmentation-aware refinement is crucial for translating high-quality, boundary-aware semantics from 2D foundation models to 3D.
- A two-stage approach, separating per-shape teacher field optimization from generalizable feedforward prediction, effectively transfers knowledge and achieves real-time inference while maintaining feature quality and rotation robustness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.