Reproduce “Online Zero-Shot Classification with CLIP” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qi Qian, Juhua Hu
OnZeta, a framework for online zero-shot image classification, enables CLIP to adapt to streaming data without storing image representations. It achieved over 3% average accuracy improvement compared to baseline CLIP across diverse datasets and outperformed existing test-time prompt tuning methods while maintaining computational efficiency for real-time applications.
Key insights
- Zero-shot classification performance can be substantially improved in an online setting by dynamically adapting class proxies and pseudo-labels without requiring data storage or iterative passes.
- The synergistic combination of online distribution-aware label refinement and vision-space proxy learning effectively addresses both class imbalance and the modality gap.
- Theoretical convergence guarantees support the stability and effectiveness of the proposed online learning algorithms for continuous data streams.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.