Reproduce “StarHarness: Evolving Harnesses with Stratified Search for Enterprise Environments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Esakkivel Esakkiraja, Denis Akhiyarov, Vikas Yadav, Sai Rajeswar, Patrice Bechard, Sridhar Nemala, Sagar Davasam
StarHarness introduces an automated framework for evolving LLM agent harnesses, enhancing performance in complex, tool-rich enterprise environments without modifying the underlying language model. The system achieved task success gains of up to +22.3 percentage points on enterprise benchmarks and demonstrated effective cross-model transferability while reducing inference costs by up to 53%.
Key insights
- Significant performance improvements for fixed LLM agents in enterprise settings can be achieved by evolving their external harnesses, effectively bridging the "model-environment mismatch."
- The learned harness modifications generalize to unseen tasks and successfully transfer across different LLM models (e.g., GPT and Qwen families) without requiring re-evolution.
- Harness evolution primarily addresses three types of model-environment friction: repairing interfaces, making implicit environment conventions explicit, and compressing operational knowledge into deterministic components.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.