Reproduce “Instance-aware Semantic Segmentation via Multi-task Network Cascades” on OpenResearch

About this paper

Jifeng Dai, Kaiming He, Jian Sun

Multi-task Network Cascades (MNCs) are introduced for instance-aware semantic segmentation, decomposing the task into three causally linked stages: box proposal, mask prediction, and category classification within an end-to-end trainable convolutional neural network. The method achieved state-of-the-art results on PASCAL VOC and MS COCO, notably improving accuracy (e.g., 63.5% [email protected] on PASCAL VOC) and dramatically increasing inference speed to 360ms per image.

Key insights

• Decomposing complex instance-aware segmentation into a causal cascade of simpler, inter-dependent deep learning tasks (box, mask, category) enables more effective optimization and feature learning.
• The ability to train a causally dependent network cascade end-to-end, specifically with differentiable mechanisms for spatial transformations based on predicted outputs, is crucial for unifying and optimizing the entire instance segmentation pipeline.
• Integrating the region proposal generation directly within a shared-feature CNN eliminates external, slow modules, leading to a significant increase in inference speed without compromising accuracy.

Summary by alphaXiv.