Reproduce “OV-VG: A Benchmark for Open-Vocabulary Visual Grounding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chunlei Wang, Wenquan Feng, Xiangtai Li, Guangliang Cheng, Shuchang Lyu, Binghao Liu, Lijiang Chen, Qi Zhao
A new benchmark, OV-VG, along with a dedicated framework for open-vocabulary visual grounding, is introduced to evaluate the precise localization of novel objects using detailed language descriptions. The proposed approach achieves state-of-the-art performance on this benchmark, improving average Acc50 by 2.43% compared to existing methods, even when leveraging extensive pre-training.
Key insights
- Truly open-vocabulary visual grounding of novel concepts is a difficult challenge for current state-of-the-art models, which often struggle with precise localization without prior exposure.
- Data leakage from large pre-training datasets significantly impacts reported performance in open-vocabulary tasks, highlighting the need for carefully controlled benchmarks to assess genuine generalization.
- Modular components like LGFA and TIQS, which improve attention to text-relevant regions and select linguistically pertinent proposals, are effective in enhancing visual-linguistic alignment and novel category recognition for OV-VG.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.