Reproduce “AMU-Tuning: Effective Logit Bias for CLIP-based Few-shot Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuwei Tang, Zhenyi Lin, Qilong Wang, Pengfei Zhu, Qinghua Hu
Researchers from Tianjin University developed AMU-Tuning, a method that enhances CLIP's few-shot learning performance by systematically designing an effective logit bias. This approach achieves state-of-the-art accuracy on ImageNet-1K and other datasets, while significantly reducing computational costs compared to prior methods.
Key insights
- For generating effective logit bias, the "complementarity" of auxiliary features (their difference from zero-shot CLIP) is more critical than their individual performance.
- Optimizing the logit bias predictor separately through multi-branch training and initializing it with class-wise auxiliary features significantly enhances performance.
- Adaptively fusing the logit bias based on the zero-shot CLIP's prediction uncertainty (e.g., using Kurtosis) improves robustness and accuracy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.