Reproduce “PART: Learning 3D Part Assembly and Retrieval with Transformers” on OpenResearch

About this paper

Ruchao Bao, Wenzheng Wu, Chucheng Xiang, Zhongyuan Liu, Yuan Liu, Jinxin Dong, Ligang Liu, Ziqi Wang

PART selects reusable components from a candidate-part library to assemble a target point cloud, predicting each part’s identity, rigid pose, and target-point segmentation with a variable-length set of learned queries. Test-time optimization refines poses and segmentation jointly. On PartNet Chair, its Shape Chamfer Distance is 1.84 versus 4.46 for UPRA and 8.27 for a genetic algorithm, but it does not model collisions, connections, or structural stability.

Key insights

• Target segmentation and part pose estimation reinforce one another: predicted regions constrain placement, while transformed parts provide an independent explanation of those regions.
• The A-Matrix rotation representation and explicit handling of 24-fold cubic symmetries reduce ambiguity and instability in orientation learning.
• Training with short libraries while testing on much larger libraries allows the same model to handle up to 1,000 candidates, though attention memory grows with library size.

Summary by alphaXiv.