Reproduce “Harness Engineering for Agentic AI Coding Tools: An Exploratory Study” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Matthias Galster, Seyedmoein Mohsenimofidi, Jai Lal Lulla, Muhammad Auwal Abubakar, Christoph Treude, Sebastian Baltes
Harness engineering broadens the study of agentic coding-tool configuration beyond instruction files to include skills, subagents, commands, rules, settings, hooks, and external integrations. An analysis of 32,564 engineered GitHub repositories found that adoption remains concentrated on Markdown Context Files, while executable and extensible mechanisms are comparatively rare; 2,853 repositories contained at least one relevant configuration artifact.
Key insights
- Configuration practice is dominated by Context Files: they occur in 2,586 of 2,853 configured repositories, or 90.6%, whereas Skills and Subagents appear in only 158 and 131 repositories.
- Most repositories configure a single tool, but `AGENTS.md` is emerging as a tool-agnostic shared document; 301 references from `CLAUDE.md` to `AGENTS.md` indicate growing convergence around the format.
- Skills are generally structured text rather than executable extensions: 514 of 601 Skills, or 85.5%, contain no scripts, references, or assets, and persistent Subagent memory was not observed.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.