Reproduce “PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment” on OpenResearch

About this paper

Kaixin Wang, Jun Hao Liew, Yingtian Zou, Daquan Zhou, Jiashi Feng

A Prototype Alignment Network (PANet) was developed by researchers at the National University of Singapore for few-shot semantic segmentation. This model leverages metric learning with class-specific prototypes and a novel Prototype Alignment Regularization (PAR) to establish new state-of-the-art mean-IoU scores on PASCAL-5i (48.1% for 1-shot, 55.7% for 5-shot) and exhibits robustness to weak annotations.

Key insights

• A non-parametric metric learning framework, separating prototype learning from dense prediction, is an effective strategy for few-shot semantic segmentation.
• Explicitly enforcing consistency between prototypes derived from query predictions and support images (via PAR) significantly improves model generalization and effective utilization of limited support data.
• The model's design, including a late fusion strategy, enables robust performance even when trained with weaker annotations (e.g., scribbles, bounding boxes) on the support set.

Summary by alphaXiv.