Reproduce “Context Patch Fusion With Class Token Enhancement for Weakly Supervised Semantic Segmentation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yiyang Fu, Hui Li, Wangyu Wu

The CPF-CTE framework, developed by researchers from Wuxi University, Xiamen University, and the University of Liverpool, introduces a novel approach for Weakly Supervised Semantic Segmentation. It integrates a Contextual-Fusion BiLSTM and post-hoc learnable class token enhancement to address spatial discontinuity and refine class-specific semantics in Transformer-based methods, achieving new state-of-the-art performance with 69.5% mIoU on PASCAL VOC 2012 and 45.4% mIoU on MS COCO 2014.

Key insights

  • The CF-BiLSTM effectively restores spatial continuity and models bidirectional contextual dependencies among ViT patches, thereby enriching feature representations and mitigating patch discontinuity.
  • Introducing learnable class tokens after the ViT encoder, rather than early in the attention layers, prevents class competition and allows for cleaner, more effective class-specific semantic refinement.
  • The synergistic combination of explicit spatial context fusion and refined class-specific guidance leads to superior pseudo-label quality, which is crucial for overall WSSS performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.