Reproduce “Improving FIM Code Completions via Context & Curriculum Based Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hitesh Sagtani, Rishabh Mehrotra, Beyang Liu
Researchers from Sourcegraph and Pavo AI developed a combined curriculum and context-aware learning approach to enhance Fill-in-the-Middle (FIM) code completion models. This method significantly improved completion acceptance and persistence rates, particularly for smaller language models, and introduced a new multi-line infilling benchmark for more realistic evaluation.
Key insights
- Combining curriculum learning and context-aware training provides complementary benefits, leading to greater performance improvements than either strategy in isolation.
- Smaller language models (1B-7B parameters) show more pronounced improvements from this fine-tuning approach, enhancing their viability for latency-constrained, real-time coding environments.
- Targeting specific "hard-to-complete" Abstract Syntax Tree (AST) node types through curriculum learning directly translates to improved real-world completion acceptance rates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.