Reproduce “SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiayi Liu, Denys Iliash, Angel X. Chang, Manolis Savva, Ali Mahdavi-Amiri
SINGAPO presents a diffusion-based framework to generate 3D articulated objects from a single RGB image, inferring part connectivity and articulation parameters to create visually consistent and kinematically plausible models. The method outperforms existing baselines in geometric accuracy and reduces part collisions, with a user study confirming higher realism of generated assets.
Key insights
- Decomposing the generation into graph inference, abstract attribute generation, and mesh assembly effectively addresses the ambiguities of single-image input while providing a modular and interpretable pipeline.
- The integration of a novel Image Cross-Attention (ICA) module and graph guidance within a transformer-based diffusion model is crucial for enforcing geometric consistency with the input image and kinematic plausibility among parts.
- Classifier-free training with random dropping of object category, graph, and image guidance significantly enhances the model's generalization capabilities across diverse articulated objects.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.