Reproduce “Robust Instance Segmentation through Reasoning about Multi-Object Occlusion” on OpenResearch

About this paper

Xiaoding Yuan, Adam Kortylewski, Yihong Sun, Alan Yuille

Researchers at Tongji University and Johns Hopkins University developed a deep network with an Occlusion Reasoning Module (ORM) that performs robust multi-object instance segmentation under occlusion, trained exclusively with bounding box supervision. This approach significantly improved modal instance segmentation by 9.6% mIoU for 30%-60% occluded objects and 11.3% mIoU for 60%-90% occluded objects on the KINS dataset, and achieved 76.7% amodal mIoU on KINS, outperforming existing weakly and self-supervised methods.

Key insights

• Explicitly reasoning about mutual object relationships and estimating a consistent occlusion order are crucial for resolving ambiguities and inconsistencies in multi-object instance segmentation.
• A self-correction mechanism that updates segmentation likelihoods based on recovered occlusion order allows for top-down refinement and improved classification of occluded objects.
• Robust modal and amodal instance segmentation can be achieved under weak supervision (bounding boxes only) by integrating generative models with dedicated occlusion reasoning capabilities.

Summary by alphaXiv.