Reproduce “ViewCo: Discovering Text-Supervised Segmentation Masks via Multi-View Semantic Consistency” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pengzhen Ren, Changlin Li, Hang Xu, Yi Zhu, Guangrun Wang, Jianzhuang Liu, Xiaojun Chang, Xiaodan Liang
Addressing the rigid image-text correspondence and ambiguity in textual supervision, ViewCo introduces a multi-view consistent learning framework for text-supervised semantic segmentation. This method leverages text-to-views and cross-view segmentation consistency to achieve improved zero-shot mIoU scores on benchmarks like PASCAL VOC 2012, reaching 52.4%, and boosts image classification performance.
Key insights
- Enforcing segment-level visual consistency across augmented image views provides an intrinsic self-supervisory signal, stabilizing segmentation masks against ambiguous text supervision.
- Relaxing image-text correspondence to a one-to-many text-to-views relationship significantly enhances high-level cross-modal semantic alignment.
- Combining these multi-view consistency schemes yields a more robust and generalized model for learning segmentation masks without dense annotations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.