Reproduce “RELOCATE: A Simple Training-Free Baseline for Visual Query Localization Using Region-Based Representations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Savya Khosla, Sethuraman T V, Alexander Schwing, Derek Hoiem
Researchers at the University of Illinois Urbana-Champaign developed RELOCATE, a training-free baseline for Visual Query Localization that leverages pre-trained SAM and DINO models within a refined stage-wise pipeline. This method achieves state-of-the-art performance on the Ego4D VQ2D dataset, with a 49% relative improvement in spatio-temporal average precision (stAP25) over previous methods.
Key insights
- Powerful pre-trained vision models like SAM and DINO can be effectively orchestrated within a modular pipeline to achieve state-of-the-art VQL without task-specific training.
- Representing video content as compact, region-based object tokens is significantly more efficient than patch-based methods for long videos while maintaining semantic richness.
- Object-centric refinement and query expansion through reiteration are crucial components for improving spatial accuracy, filtering spurious matches, and handling appearance variations over time.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.