Reproduce “Understanding Complexity in VideoQA via Visual Program Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Cristobal Eyzaguirre, Igor Vasiljevic, Achal Dave, Jiajun Wu, Rares Andrei Ambrus, Thomas Kollar, Juan Carlos Niebles, Pavel Tokmakov
A data-driven method quantifies query complexity in VideoQA by analyzing the structural and semantic intricacy of generated visual programs. This approach, called CodePlexity, reveals specific reasoning patterns challenging current models and enables the automatic generation of a new benchmark, CodePlex-QA, which is 1.9 times harder than NExT-QA.
Key insights
- Human intuition is unreliable for assessing VideoQA question difficulty for AI models; questions perceived as average by humans often prove most challenging for models.
- The complexity of generated visual programs, particularly as measured by CodePlexity, provides a robust and generalizable proxy for actual model performance challenges in VideoQA.
- Current VideoQA models universally struggle with fine-grained temporal reasoning and detailed spatio-temporal object-centric analysis within video content.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.