Reproduce “CODESTRUCT: Code Agents over Structured Action Spaces” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Myeongsoo Kim, Joe Hsu, Dingmin Wang, Shweta Garg, Varun Kumar, Murali Krishna Ramanathan

CODESTRUCT introduces a framework enabling large language model (LLM) agents to interact with source code repositories using a structured action space derived from Abstract Syntax Trees (ASTs), moving beyond traditional text-based operations. The system demonstrates improved task completion accuracy on benchmarks like SWE-Bench Verified by up to 20.8% for smaller models, concurrently reducing token consumption and inference costs.

Key insights

  • Directly interacting with code through AST-grounded actions substantially reduces the brittleness and error rates inherent in text-based code manipulation by LLM agents, enhancing reliability.
  • Structured action spaces significantly improve agent efficiency, leading to substantial reductions in token consumption, API calls, and overall inference costs for code-related tasks.
  • Smaller LLMs exhibit disproportionately large gains in task completion accuracy when provided with structure-aware interfaces, suggesting this paradigm unlocks greater capabilities for less powerful models.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.