Reproduce “MMEnterprise-Bench: Evaluating Enterprise Agents over Evolving Multimodal Records” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Seohyun Lee, Jongha Kim, Dohwan Ko, Ji Soo Lee, Seoung Choi, Jayun Ku, Hyunwoo J. Kim

MMEnterprise-Bench tests whether agents can answer questions about evolving workplace records across chat, documents, and images. It uses 1,613 questions from 30 simulated projects, retaining earlier records to test evidence reasoning, conflict resolution, and unanswerability. Across ten multimodal models, GPT-5.6 Sol reaches 68.0% accuracy, and Qwen3.5-27B leads open-source models at 50.3% with 50 retrieved records. These benchmark results do not establish reliability in real enterprise settings.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.