Reproduce “Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhengyu Chen, Teng Xiao, Huaisheng Zhu, Yige Yuan, Luan Zhang, Jingang Wang
Co-Harness improves an agent’s runtime harness alongside its model by using an LLM to diagnose failed tool-integrated reasoning runs, validate configuration repairs, and fine-tune the model on successful trajectories. Across Qwen3-8B and Qwen3-32B on three math benchmarks, Round 2 reached 78.9% mean pass@1 versus 58.5% at Round 0, but the combined schedule does not isolate causal contributions.
Key insights
- Jointly optimizing the agent's Harness and model parameters in an alternating fashion creates a compounding self-improvement mechanism.
- An LLM-based critic can effectively identify Harness-level failure modes and autonomously propose validated, localized updates to various Harness components.
- Distilling behaviors from an improved Harness into model weights through fine-tuning enables the model to internalize better reasoning and tool-use strategies, reducing reliance on external scaffolding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.