Reproduce “RSVG-ZeroOV: Exploring a Training-Free Framework for Zero-Shot Open-Vocabulary Visual Grounding in Remote Sensing Images” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ke Li, Di Wang, Ting Wang, Fuyu Dong, Yiming Zhang, Luyao Zhang, Xiangyu Wang, Shaofeng Li and 1 more
RSVG-ZeroOV introduces a training-free framework for zero-shot open-vocabulary visual grounding in remote sensing images by synergistically integrating frozen Vision-Language Models and Diffusion Models. This approach significantly outperforms existing methods on RSREC and RSRES benchmarks without requiring any task-specific training data.
Key insights
- Generic VLMs possess strong generalization capabilities for unseen remote sensing data, and Diffusion Models excel at perceiving inherent structural concepts of objects.
- Cross-attention maps from VLMs and self-attention maps from DMs are complementary, and their intelligent interaction significantly improves visual grounding performance.
- The specific "Overview-Focus-Evolve" order for combining VLM semantics, DM structure, and attention refinement is crucial for accurate and robust zero-shot open-vocabulary RSVG.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.