Reproduce “SK-Net: Deep Learning on Point Cloud via End-to-end Discovery of Spatial Keypoints” on OpenResearch

About this paper

Weikun Wu, Yan Zhang, David Wang, Yunqi Lei

SK-Net introduces an end-to-end deep learning framework that autonomously discovers spatial keypoints (Skeypoints) within 3D point clouds, simultaneously learning comprehensive feature representations for tasks like classification and segmentation. This method achieved 95.8% accuracy on ModelNet10 and an 85.0% mIoU for part segmentation on ShapeNetPart, demonstrating robustness to data corruption and efficient feature representation.

Key insights

• Jointly optimizing the unsupervised discovery of discriminative Skeypoints with task-specific feature learning offers a more principled and effective approach to 3D shape understanding.
• The integration of both local detail features and broader spatial pattern features, mediated by learnable Skeypoints, is crucial for developing a comprehensive and robust 'Pattern-Detail' representation of point clouds.
• The combination of a task-specific loss with unsupervised regulating losses can effectively guide a deep network to autonomously generate meaningful geometric primitives that enhance downstream task performance without explicit supervision.

Summary by alphaXiv.