Reproduce “Scan2CAD: Learning CAD Model Alignment in RGB-D Scans” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Armen Avetisyan, Manuel Dahnert, Angela Dai, Manolis Savva, Angel X. Chang, Matthias Nießner
This work introduces the Scan2CAD dataset, the first large-scale collection of fine-grained 9 degrees-of-freedom alignments between real-world RGB-D scans and synthetic CAD models. It also proposes a learning-based method using a 3D CNN and variational optimization to align clean CAD models to noisy scan geometry, achieving 31.68% alignment accuracy and outperforming prior methods.
Key insights
- A dedicated, large-scale dataset with fine-grained cross-domain alignments is essential for training robust learning-based solutions for scan-to-CAD alignment.
- A multi-task 3D CNN, predicting correspondence heatmaps, semantic compatibility, and scale, effectively learns joint embeddings between noisy scan geometry and pristine CAD models.
- Integrating a variational optimization framework with learned correspondence heatmaps allows for accurate 9 degrees-of-freedom pose recovery, even amidst geometric disparities between domains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.