Reproduce “TimeProVe: Propose, then Verify for Efficient Long Video Temporal Reasoning in Activities of Daily Living” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Arkaprava Sinha, Dominick Reilly, Siddharth Krishnan, Hieu Le, Srijan Das
A new hybrid framework, TIMEPROVE, enhances long video question answering by using lightweight models to propose temporal evidence clips before targeted verification with expensive Vision-Language Models. This "propose, then verify" approach reduces VLM invocations by 75% and inference costs by 93%, achieving a 45.1% accuracy on the new OPENTSUBENCH benchmark for real-world Activities of Daily Living videos.
Key insights
- Decoupling broad temporal proposal generation with lightweight models from targeted VLM verification offers substantial computational efficiency (up to 93% cost reduction) while maintaining accuracy for long video reasoning.
- Grounding evidence hypotheses in learned action priors, rather than sparse captions, is critical for preserving fine-grained, temporally localized visual details necessary for complex Activities of Daily Living queries.
- The novel OPENTSUBENCH benchmark addresses limitations of existing datasets by requiring open-ended, temporally grounded answers for long ADL videos, driving more robust and interpretable model development.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.