Reproduce “SimuVerity: Benchmarking Agents for Engineering-Grade Simulink Model Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ruiqi Zhang, Jiahao Wang, Mingxuan Li, Haichen Luo, Chaoting Wang, Guoyu Mou, Keyu Lai, Hanchao Lv and 4 more

SimuVerity tests generated Simulink models against task-specific engineering requirements in native simulations, rather than judging resemblance to a reference diagram. The authors introduce 101 tasks across ten domains and assess six agent systems using delivery, execution, qualification, and performance measures. Claude Opus 4.8 with Claude Code has the highest mean score, 42.86, but is not statistically distinguishable from GPT-5.5 with Codex.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.