Reproduce “The Right Prompts for the Job: Repair Code-Review Defects with Large Language Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zelin Zhao, Zhaogui Xu, Jialong Zhu, Peng Di, Yuan Yao, Xiaoxing Ma
Researchers from Ant Group and Nanjing University investigated leveraging Large Language Models (LLMs) to repair code-review defects. Their work demonstrated that finetuned models like CodeLLaMA can achieve up to 72.97% exact code match by effectively utilizing review comments and precise bug location information, highlighting the importance of comprehensive prompt engineering and diverse training datasets for automated program repair.
Key insights
- Contextual information, especially review comments and precise fix ranges, are critical for maximizing LLM performance in code defect repair, improving ECM by over 20% from a minimal prompt.
- Finetuning LLMs significantly outperforms zero-shot approaches for this task, with models around 6-7 billion parameters offering an optimal balance between effectiveness and computational efficiency.
- Defect characteristics and comment types differ substantially between human-generated and automated-checker datasets, necessitating training on diverse data for comprehensive repair capabilities due to poor cross-dataset transferability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.