Reproduce “Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Davy Neven, Bert De Brabandere, Marc Proesmans, Luc Van Gool
A method for instance segmentation achieves real-time performance and high-resolution masks by jointly optimizing spatial embeddings with instance-specific clustering bandwidths, leveraging a Lovasz-hinge loss. This approach reached 27.6 AP on the Cityscapes benchmark at 11 frames per second.
Key insights
- Learning instance-specific clustering bandwidths significantly improves segmentation accuracy (10.7 AP gain in ablation) by allowing adaptive margins that better accommodate diverse object sizes and shapes.
- Directly optimizing for IoU via the Lovasz-hinge loss enables effective end-to-end training of embedding-based instance segmentation, overcoming limitations of separate regression and clustering steps.
- Allowing the network to learn the optimal center of attraction and utilizing elliptical margins consistently provides further performance improvements, demonstrating the benefits of increased flexibility in the embedding space.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.