Reproduce “Training-Free Class Purification for Open-Vocabulary Semantic Segmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qi Chen, Lingxiao Yang, Yun Chen, Nailong Zhao, Jianhuang Lai, Jie Shao, Xiaohua Xie
Researchers from Sun Yat-sen University and ByteDance introduced FreeCP, a training-free framework that purifies semantic categories in open-vocabulary semantic segmentation by mitigating class redundancy and visual-language ambiguity. This plug-and-play module consistently boosts the performance of existing training-free methods, for example, improving MaskCLIP's mIoU by 10.9% and SCLIP's mIoU by 3.9% across various benchmarks.
Key insights
- Refining class activation maps alone without purification can degrade performance in open-vocabulary settings by reinforcing irrelevant activations.
- Spatial consistency (IoU between activation maps) serves as a reliable cue to diagnose both class redundancy (low intra-class consistency) and visual-language ambiguity (high inter-class consistency).
- The order of purification matters; addressing class redundancy before tackling visual-language ambiguity is crucial for effective performance improvement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.