Reproduce “WorkflowArena: Benchmarking Executable Workflow Orchestration in LLM-Based Multi-Agent Systems” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zheng Jiang, Fang Niu, Bo Peng, Houde Qian, Wenyi Li, Deyao Hong, Mingju Gao, Yizhe Chi and 5 more
WorkflowArena evaluates how models organize long-horizon tasks by choosing whether and how to delegate, recording workflows alongside deliverables across 145 tasks in eight domains. In a fixed-executor transfer test, Kimi scores 45.04 with static workflows authored by Opus and 29.69 with those authored by Astra. Overall Resolved peaks at 36.95%, an equal-weight domain average; no configuration leads every domain.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.