Reproduce “ASTER: Natural and Multi-language Unit Test Generation with LLMs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Rangeet Pan, Myeongsoo Kim, Rahul Krishna, Raju Pavuluri, Saurabh Sinha

ASTER, a framework from IBM Research, integrates Large Language Models (LLMs) with static program analysis to generate natural and multi-language unit tests. It achieved significantly higher code coverage for Java EE and Python applications compared to traditional tools, with developers expressing high willingness to adopt its generated tests.

Key insights

  • Integrating LLM code synthesis with structured program analysis enables the generation of high-quality, 'natural' unit tests that are both high in coverage and developer-friendly.
  • The ability of LLMs to understand diverse programming languages and leverage mocking contexts facilitates effective test generation for complex, multi-language, and enterprise-grade applications.
  • Even smaller, more resource-efficient LLMs can achieve competitive test generation performance compared to larger models, making LLM-based ATG more practical and cost-effective.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.