Reproduce “TDAD: Test-Driven Agentic Development - Reducing Code Regressions in AI Coding Agents via Graph-Based Impact Analysis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pepe Alonso, Sergio Yovine, Victor A. Braberman
This research introduces TDAD, an open-source methodology and tool that uses graph-based impact analysis to integrate structural code knowledge into AI coding agents. The approach significantly reduced test-level regressions by 70% and, in some configurations, improved issue resolution rates, by enabling agents to identify and self-correct potential regressions.
Key insights
- Providing concise, targeted structural knowledge (graph-based impact analysis) to AI agents is more effective for reducing regressions than verbose procedural instructions (e.g., generic TDD prompts), which can paradoxically increase regressions.
- Elevating regression rate to a first-class metric, alongside resolution rate, is crucial for developing more robust and practically useful AI coding agents, as it incentivizes stable and reliable code generation.
- Graph-based code-test dependency analysis not only helps prevent regressions but also improves agents' fundamental ability to understand codebases, navigate them, and generate effective patches, thus boosting resolution rates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.