Reproduce “VSAS-Bench: Real-Time Evaluation of Visual Streaming Assistant Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pavan Kumar Anasosalu Vasu, Cem Koc, Fartash Faghri, Chun-Liang Li, Bo Feng, Zhengfeng Lai, Meng Cao, Oncel Tuzel and 1 more
The authors introduce VSAS-Bench, a benchmark for evaluating visual assistants on continuing video streams. It compares synchronous evaluation with asynchronous camera input and adapts conventional video models using an external memory buffer without additional model training. Adapted Qwen3-VL-4B scores 46.8 overall asynchronous accuracy versus Dispider’s 43.4, but scores 17.9 on Future versus Dispider’s 53.1, limiting the aggregate advantage to this benchmark.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.