Reproduce “CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Grounding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Linhui Xiao, Xiaoshan Yang, Fang Peng, Ming Yan, Yaowei Wang, Changsheng Xu
A research group from CASIA and Alibaba's DAMO Academy introduces CLIP-VG, an unsupervised visual grounding framework that leverages CLIP and self-paced curriculum learning to effectively manage noisy pseudo-labels. The method achieves state-of-the-art performance for unsupervised visual grounding, notably surpassing weakly supervised approaches and competing with fully supervised models while dramatically reducing computational costs.
Key insights
- Self-paced curriculum learning effectively filters noisy pseudo-labels, significantly improving generalization and robustness in unsupervised visual grounding.
- Pre-trained vision-language models like CLIP can be efficiently adapted for fine-grained, region-level tasks by freezing their encoders and using a lightweight cross-modal transformer.
- Intelligently combining multiple disparate pseudo-label sources through a structured curriculum enhances linguistic and semantic diversity, leading to stronger overall model performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.