Reproduce “OR-Space: A Full-Lifecycle Workspace Benchmark for Industrial Optimization Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chenyu Zhou, Xinyun Lu, Jiangyue Zhao, Jianghao Lin, Dongdong Ge, Yinyu Ye

OR-Space introduces a full-lifecycle workspace benchmark for evaluating LLM agents on industrial optimization tasks, moving beyond simplified problem statements. It reveals that even frontier models struggle with workspace-grounded model construction and exhibit specific failure modes in data mapping and constraint interpretation across multiple artifacts, while also highlighting the distinct challenge of generating grounded explanations.

Key insights

  • Workspace-grounded model construction and iterative revision remain significant challenges for frontier LLM agents, requiring complex data schema recovery and semantic alignment across heterogeneous artifacts.
  • The benchmark exposes novel failure modes, such as data-mapping errors, hallucinated constraints, and "context poisoning" from legacy code, which are often concealed in text-only evaluations.
  • Grounded explanation tasks assess distinct capabilities in multi-hop reasoning over diverse artifacts and solver outputs, showing weak correlation with model construction abilities.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.