Reproduce “VLOD-TTA: Test-Time Adaptation of Vision-Language Object Detectors” on OpenResearch

About this paper

Atif Belal, Heitor R. Medeiros, Marco Pedersoli, Eric Granger

Researchers at LIVIA, ETS Montréal, developed VLOD-TTA, an efficient Test-Time Adaptation (TTA) framework for Vision-Language Object Detectors (VLODs). It improves model robustness under test-time distribution shifts by leveraging IoU-weighted Entropy Minimization (IWE) and Image-Conditioned Prompt Selection (IPS), achieving average mAP gains of +3.3 on stylized datasets and comparable or better mAP on common corruptions while significantly reducing computational overhead compared to prior TTA methods.

Key insights

• Incorporating spatial coherence of object proposals through IoU-weighting during adaptation significantly improves robustness and reduces confirmation bias in object detection TTA.
• Image-conditioned selection of text prompts, rather than uniform averaging, leads to more effective and context-relevant adaptation for Vision-Language Object Detectors.
• Efficient TTA for VLODs can be achieved by fine-tuning only a small set of adapter parameters, maintaining low computational overhead while generalizing across diverse VLOD architectures (CNN- and transformer-based).

Summary by alphaXiv.