Reproduce “Bayesian Test-time Adaptation for Object Recognition and Detection with Vision-language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lihua Zhou, Mao Ye, Shuaifeng Li, Nianxin Li, Jinlin Wu, Xiatian Zhu, Lei Deng, Hongbin Liu and 2 more
The paper introduces BCA+, a training-free framework for adapting Vision-Language Models (VLMs) during inference, unifying object recognition and open-vocabulary object detection. This approach integrates dynamic likelihood and prior adaptation within a Bayesian framework and achieves state-of-the-art performance on various out-of-distribution and cross-domain benchmarks, for example, improving mAP50 by 3.45% on COCO-C compared to Grounding DINO.
Key insights
- Dynamic adaptation of both likelihood and class prior distribution is crucial for robust VLM performance under real-world distribution shifts, as static priors lead to suboptimal predictions.
- A training-free, cache-based Bayesian approach can effectively unify test-time adaptation for both object recognition and open-vocabulary object detection.
- Intelligent fusion of initial VLM predictions with adaptive cache-based predictions, guided by prediction uncertainty, yields superior and more robust performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.