Reproduce “From Question Answering to Task Completion: A Survey on Agent System and Harness Design” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jianyuan Guo, Zhiwei Hao, Chengcheng Wang, Cheng Fan, Tingzhang Luo, Hongguang Li, Ying Gao, Hefei Mei and 9 more
The survey explains how agent performance depends on models working through runtimes tailored to task demands. It organizes agent engineering into four coexisting paradigms and describes six runtime responsibilities spanning observation, context, control, action, state, and verification. On Terminal-Bench 2.0, GPT-5.3-Codex scores 64.7% with Terminus 2 and 78.4% with SageAgent; reported pairings vary, without establishing causation.
Key insights
- Agent performance is an emergent property resulting from the intricate interaction between the foundation model's capabilities, the execution harness design, the task structure, and the evaluation methodology.
- The execution harness is a fundamental, coupled runtime system, not merely an optional wrapper, with its six components profoundly influencing each other's effectiveness.
- Effective harness design is highly context-dependent, with specific task properties (e.g., horizon, environment type, autonomy level) dictating the optimal configuration of harness components.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.