Reproduce “Teaching Code LLMs to Use Autocompletion Tools in Repository-Level Code Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chong Wang, Jian Zhang, Yebo Feng, Tianlin Li, Weisong Sun, Yang Liu, Xin Peng
TOOLGEN, a system from Nanyang Technological University and Fudan University, integrates program-analysis-based autocompletion tools into large language models for repository-level code generation. This integration significantly improves the static validity of generated code by 44.9% to 57.7% and enhances functional correctness by up to 40%, directly addressing dependency errors.
Key insights
- LLMs can be effectively trained to learn and predict specific points within a code sequence where external tool invocation for autocompletion is beneficial.
- A constrained decoding strategy, leveraging a prefix tree of tool suggestions, enables LLMs to intelligently select the correct completion from multiple options provided by program analysis tools.
- Strategic integration of program analysis tools significantly enhances the contextual correctness and validity of LLM-generated code for repository-level tasks without compromising overall code quality or efficiency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.