Reproduce “Bayesian Prompt Learning for Image-Language Model Generalization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mohammad Mahdi Derakhshani, Enrique Sanchez, Adrian Bulat, Victor Guilherme Turrisi da Costa, Cees G. M. Snoek, Georgios Tzimiropoulos, Brais Martinez
Researchers at the University of Amsterdam and Samsung AI Cambridge introduced Bayesian Prompt Learning (BPL), a new approach that frames prompt learning as a variational inference problem to regularize the prompt space. This method consistently improved generalization performance on unseen prompts, cross-dataset, and cross-domain tasks compared to existing ERM-based prompt learning techniques, achieving an average accuracy of 74.94% for unseen prompt generalization.
Key insights
- Framing prompt learning as a variational inference problem provides an inherent regularization mechanism, preventing overfitting to specific training prompts and encouraging the learning of transferable invariant features.
- Modeling the prompt space probabilistically allows the system to cover a broader range of prompt variations and capture fine-grained factors of variation within classes, enhancing robustness.
- Ensembling multiple sampled prompts from the learned variational distribution during inference significantly boosts generalization performance compared to relying on a single deterministic prompt or just the mean of the distribution.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.