Reproduce “MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mattia Soldan, Alejandro Pardo, Juan León Alcázar, Fabian Caba Heilbron, Chen Zhao, Silvio Giancola, Bernard Ghanem
The MAD dataset introduces a new benchmark for temporal language grounding, leveraging professionally curated movie audio descriptions to create a large-scale, bias-reduced resource for long-form untrimmed videos. It reveals existing model limitations in processing extended video content and demonstrates the value of scalable, high-quality data collection for improving both grounding and retrieval tasks.
Key insights
- Professionally generated movie audio descriptions offer a high-quality, scalable, and less biased source for creating large-scale video-language grounding datasets.
- Current state-of-the-art video grounding models face substantial challenges when applied to long-form, untrimmed videos, showing significantly reduced performance compared to short-clip scenarios.
- The sheer scale of automatically collected, even potentially noisy, data can compensate for inherent imperfections, leading to improved performance in both video grounding and text-to-video retrieval tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.