Reproduce “An Empirical Study of Agent Developer Practices in AI Agent Frameworks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yanlin Wang, Xinyi Xu, Jiachi Chen, Tingting Bi, Wenchao Gu, Zibin Zheng
An empirical study analyzed thousands of GitHub repositories and developer discussions to map the landscape of LLM-based agent framework adoption, identify recurring challenges across the software development lifecycle, and compare how different frameworks address developer needs. The investigation found that most projects combine multiple frameworks and detailed a taxonomy of prevalent issues related to agent logic, tool integration, performance, and version compatibility.
Key insights
- Most top-starred agent projects rely on a combination of two or more frameworks, indicating the complexity of real-world agent development tasks.
- A discrepancy exists between a framework's GitHub popularity and its actual sustained adoption, highlighting the importance of ecosystem maturity and maintenance over short-term trends.
- Developer challenges are multifaceted, with logic control failures (e.g., infinite loops), tool integration issues, performance bottlenecks, and version compatibility problems being particularly prevalent across the SDLC.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.