Reproduce “Take It From Here: Evaluating Coding Agents On Project Takeover” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Dingxian Wang, Zhe Fu, Jiaqi Deng, Chao Yin, Ganesh Prabhu, Karthik Gomadam Rajagopal, Darwin Yi, Huan Huo and 4 more

The authors evaluate coding-agent takeover by testing requirement reconstruction, updating, and project delivery across six configurations on 37 executable targets. Complete delivery requires passing build, requested-change, and regression checks. It averaged 4.95% with Full History and 8.11% when agents received correct target requirements, yet requested-change tests passed only 9.01%; the latter comparison changed the coding input.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.