Reproduce “IVCR-200K: A Large-Scale Multi-turn Dialogue Benchmark for Interactive Video Corpus Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ning Han, Yawen Zeng, Shaohua Long, Chengqing Li, Sijie Yang, Dun Tan, Jianfeng Dong, Jingjing Chen
Researchers introduce the Interactive Video Corpus Retrieval (IVCR) task, a multi-turn dialogue framework for video search, along with IVCR-200K, a large-scale, bilingual dataset of over 200,000 interactions. Their InterLLaVA model, built on MLLMs, achieves a video retrieval R@1 of 54.6 and moment retrieval R@1 IoU=0.5 of 19.2 on this new benchmark, demonstrating enhanced performance through multi-turn contextual learning.
Key insights
- Multi-turn dialogue context significantly enhances retrieval performance, with video retrieval R@1 improving from 38.4 at Turn 1 to 87.5 at Turn 7, highlighting the importance of accumulated historical information.
- MLLMs like InterLLaVA effectively integrate multi-modal data with multi-turn dialogue, enabling both accurate retrieval and generation of coherent, explainable natural language responses.
- High-quality, diverse, and large-scale conversational datasets are crucial for training MLLMs to handle abstract semantics and generalize across various video contents in interactive retrieval scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.