Reproduce “LeAdQA: LLM-Driven Context-Aware Temporal Grounding for Video Question Answering” on OpenResearch

About this paper

Xinxin Dong, Baoyun Peng, Haokai Ma, Yufei Wang, Zixuan Dong, Fei Hu, Xiaodong Wang

LeAdQA is a framework that enhances Multimodal Large Language Models for Video Question Answering by integrating LLM-driven causal-aware query refinement and motion-aware temporal grounding. The approach achieves state-of-the-art performance on NExT-QA, IntentQA, and NExT-GQA datasets, demonstrating improved accuracy in complex reasoning questions within long video sequences.

Key insights

• LLM-driven rephrasing of question-option pairs, which explicitly embeds causal relationships, significantly improves an MLLM's ability to perform complex causal and temporal reasoning in VideoQA.
• Precise, motion-aware temporal grounding is critical for effective video understanding, as it allows the model to focus on semantically and causally relevant content, thereby enhancing overall QA performance.
• An adaptive evidence integration strategy, including multi-threshold interval fusion and query-focused sampling, effectively manages redundancy in long videos, enabling MLLMs to efficiently process salient visual cues.

Summary by alphaXiv.