Reproduce “InstructCoder: Instruction Tuning Large Language Models for Code Editing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kaixin Li, Qisheng Hu, Xu Zhao, Hui Chen, Yuxi Xie, Tiedong Liu, Qizhe Xie, Junxian He
InstructCoder introduces the first instruction-tuning dataset for general-purpose code editing and an execution-based benchmark called EditEval. Fine-tuning open-source Large Language Models with InstructCoder enables them to achieve code editing performance comparable to proprietary models like ChatGPT.
Key insights
- Specialized instruction-tuning data, such as InstructCoder, is crucial for effective general-purpose code editing, as general instruction-tuned or zero-shot models perform poorly.
- The foundational pre-training of an LLM, particularly on a large corpus of code (e.g., Code LLaMA), significantly impacts its finetuned code editing performance.
- Dataset scaling and diversity are important; performance generally improves with a larger and more diverse set of high-quality instruction-tuning samples, and larger models benefit more from increased data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.