Reproduce “EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Thomas Hummel, Shyamgopal Karthik, Mariana-Iuliana Georgescu, Zeynep Akata
Researchers from the Tübingen AI Center, Helmholtz Munich, and TU Munich introduce EgoCVR, a new egocentric video benchmark for fine-grained composed video retrieval that demands temporal understanding. The work demonstrates that existing vision-language models struggle with action-centric modifications, while the proposed Training-Free Re-ranking CVR (TFR-CVR) method achieves 14.1% R@1 globally by leveraging large language models for query reformulation.
Key insights
- Existing state-of-the-art CVR models and general vision-language models exhibit poor performance on EgoCVR, indicating a fundamental lack of fine-grained temporal understanding in their design.
- Large Language Models (LLMs) can effectively reformulate query video captions and modification instructions into comprehensive target captions, significantly improving the quality of textual queries for retrieval.
- A modular, training-free approach that combines visual filtering with LLM-enhanced textual re-ranking is effective for complex compositional video retrieval tasks, leveraging complementary strengths of different pre-trained models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.