Reproduce “Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yangguang Li, Feng Liang, Lichen Zhao, Yufeng Cui, Wanli Ouyang, Jing Shao, Fengwei Yu, Junjie Yan
The DeCLIP framework significantly reduces the data requirements for contrastive language-image pre-training by integrating self-supervision, multi-view, and nearest-neighbor signals. It achieves comparable or superior zero-shot ImageNet accuracy with up to 7.1 times less pre-training data than existing models.
Key insights
- Widespread intrinsic supervision signals, beyond direct image-text pairing, exist within multi-modal data and can be effectively exploited to significantly improve data efficiency in pre-training.
- Combining self-supervision within modalities, multi-view supervision across modalities, and nearest-neighbor supervision leads to more robust, generalizable, and representative features, enhancing zero-shot and transfer performance.
- The proposed paradigm allows for competitive or superior performance using substantially fewer training examples and computational resources, making advanced vision-language models more accessible.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.