Reproduce “Harness Evolution Hits a Ceiling: When Weight Training Should Begin” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuan Tian, Bing Hu, Hao Wang, Binghang Lu, Fang Wu
The authors use failure labels to choose whether to evolve an agent’s harness or fine-tune its model, then test both under seed and evolved harnesses on DeepPlanning. Harness evolution raises held-out Qwen3.5-4B composite score from 0.164 to 0.297. A LoRA adapter adds 0.126 under the seed harness, while stacking it with the evolved harness adds 0.007 for 9B, within noise across four-rollout means.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.