Reproduce “HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhi Hong, Qianlong Wang, Hongchen Wei, Aidan Wu, Daohong Ying, Mingze Kong, Bei Liu, Chong Luo
HarnessFlow searches complete executable harnesses for a fixed LLM, using a tree-based policy to select evaluated candidates for full-code revision and testing. With 15 attempts per benchmark, it achieves the highest six-benchmark mean test score: 69.48%, 3.95 percentage points above Meta-Harness. This equal-weight mean reflects official benchmark scores, not pooled task instances, under the paper’s fixed model and evaluation setting.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.