Reproduce “TennisExpert: Towards Expert-Level Analytical Sports Video Understanding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhaoyu Liu, Xi Weng, Lianyu Hu, Zhe Hou, Kan Jiang, Jin Song Dong, Yang Liu
The TennisExpert framework and TennisVL benchmark enable expert-level analytical commentary for tennis videos, integrating a video semantic parser with a memory-augmented multimodal model to analyze nuanced match dynamics. This approach achieved a CIDEr score of 43.71, a five-fold increase over top-performing zero-shot models, and an LLM-based total score of 88.05, demonstrating enhanced accuracy and real-time efficiency.
Key insights
- Integrating fine-grained, structured semantic metadata from video significantly improves the accuracy and professionalism of generated sports commentary compared to raw video input.
- A hierarchical memory structure, combining short-term rally context with long-term match statistics, is crucial for MLLMs to perform comprehensive tactical reasoning and maintain coherence in analytical sports commentary.
- The proposed methodology enables real-time performance by processing video into structured representations, maintaining constant context complexity for the MLLM, and reducing computational overhead.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.