Reproduce “Termination Lag: Coding Agents Solve Early but Stop Late” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Musa Cim, Burak Topcu, Mahmut Taylan Kandemir

The authors measure when coding agents first produce a benchmark-correct patch and how long they continue afterward. They save patches after each step and test them offline with the benchmark evaluation, which the agent cannot see. For Qwen3-Coder on SWE-bench Verified, solved runs have a median post-fix lag of 16 steps, or 35% of steps; this measures steps, not time or compute savings.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.