Reproduce “Rethinking Domain Adaptation and Generalization in the Era of CLIP” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ruoyu Feng, Tao Yu, Xin Jin, Xiaoyuan Yu, Lei Xiao, Zhibo Chen

A framework rethinks domain adaptation and generalization for CLIP models by leveraging domain-specific prompts, pseudo-labeling on target data, and parameter-efficient tuning to reduce reliance on labeled source data. It demonstrates improved zero-shot recognition and enhanced generalization across various domains.

Key insights

  • CLIP's extensive pre-training provides inherently domain-robust representations, shifting the focus of domain adaptation from learning invariant features to efficient adaptation strategies.
  • Incorporating domain-specific knowledge into text prompts significantly enhances CLIP's zero-shot recognition by providing crucial contextual guidance.
  • Effective adaptation of CLIP can prioritize the utilization of unlabeled target domain data over a heavy reliance on labeled source domain data.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.