Reproduce “GARF: Learning Generalizable 3D Reassembly for Real-World Fractures” on OpenResearch

About this paper

Sihang Li, Zeyu Jiang, Grace Chen, Chenyang Xu, Siqi Tan, Xue Wang, Irving Fang, Kristof Zyskowski and 4 more

GARF is a 3D reassembly framework that generalizes from synthetic data to real-world fractures, outperforming previous methods by reducing rotation error by 82.87% and translation error by 79.83% on the Breaking Bad Everyday dataset. Developed by researchers at NYU, Yale, and Max Planck Institute, it also introduces FRACTURA, a diverse dataset of real-world and synthetic fractures, and demonstrates robust performance even with missing or extraneous fragments.

Key insights

• Learning local fracture features at scale through fracture-aware pretraining significantly enhances a model's ability to generalize across unseen object shapes and complex real-world fracture patterns.
• Leveraging Lie group theory for fragment pose modeling combined with flow matching and a multi-anchor training strategy provides a robust and precise approach to 3D reassembly.
• A two-session inference process and lightweight LoRA fine-tuning effectively boost performance and enable efficient adaptation to specific domains with limited data, addressing practical deployment challenges.

Summary by alphaXiv.