Reproduce “DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation” on OpenResearch

About this paper

Takuya Ikeda, Sergey Zakharov, Tianyi Ko, Muhammad Zubair Irshad, Robert Lee, Katherine Liu, Rares Ambrus, Koichi Nishiwaki

DiffusionNOCS estimates category-level object poses from partial observations by sampling dense NOCS maps conditioned on image and geometry cues, then registering map correspondences to depth. Its synthetic-only DINO-N model reaches 51.7 ± 8.4 mAP at 10°/5 cm on a generalization benchmark, versus 40.9 ± 10.9 for ShapePrior trained on synthetic and real data; this is an average, not a win on every dataset.

Key insights

• Diffusion models can inherently represent and resolve the multi-modality of pose estimations for symmetric objects, naturally providing a distribution of possible poses without explicit symmetry labels.
• Robust zero-shot generalization from synthetic-only training to complex real-world environments is achievable for 6D object pose estimation by combining multi-modal conditioning and diverse synthetic data augmentations.
• Flexible integration of geometric, visual, and semantic features from various modalities significantly boosts system robustness, allowing for effective pose estimation even with degraded or missing sensor data.

Summary by alphaXiv.