Reproduce “Visual Distribution Anchoring for Efficient Prompt Tuning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pouya Parsa, Raoof Zare Moayedi, Seongjin Choi
Visual Distribution Anchoring (VDA) introduces a training-free framework that adapts vision-language models to new visual domains by estimating class-level visual prototypes from unlabeled target data. This approach augments existing semantic classifiers, achieving an average gain of 3.39 percentage points on the TCP classifier across ten diverse datasets while maintaining efficient inference.
Key insights
- Text embeddings, while semantically rich, cannot effectively synthesize target-domain specific visual prototypes needed to bridge the semantic-visual gap under domain shift.
- Direct fusion of target-domain specific visual prototypes with semantic classifiers is highly effective, outperforming indirect prompt conditioning in leveraging discriminative visual signals.
- Hard assignment with confidence-based pseudo-label selection in a single pass is crucial for stable prototype estimation, avoiding issues like high-entropy collapse seen in iterative soft assignments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.