Reproduce “SPARE: Shared Perception and Adaptive Retrieval of Evidence for Streaming Video Understanding” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Senbin Xu, Zijun Zhang, Xinhua Wu, Li He, Jubo Feng, Li Qiao, Zhen Gao, Yan Zhang and 2 more

SPARE answers questions about unfolding video by checking whether recent frames suffice and retrieving earlier clips when they do not, then supplying both to an answer model trained to abstain when evidence is insufficient. With a shared visual encoder and LoRA, it scores 77.78% on OVO-Bench’s evaluated real-time and backward groups, versus 67.61% for a four-frame baseline; forward responding is excluded.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.