Reproduce “DSV-Mem: Evaluating Multimodal Memory in Professional Workflows for MLLM Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jike Zhong, Ritwick Chaudhry, Xuanbai Chen, Tianchen Zhao, Linghan Xu, Yifan Xing, Nishant Sankaran

DSV-Mem evaluates multimodal memory in professional workflows through 1,000 questions across 10 scenarios with evolving visual records and authority changes. The authors find that Gemini 3.7 Flash with medium reasoning scores 39.6% total accuracy (±0.8 across three runs), and classify 89.7% of errors across four evaluated models as state reconstruction. This researcher-constructed benchmark does not establish deployment performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.