Reproduce “CoachLM: Automatic Instruction Revisions Improve the Data Quality in LLM Instruction Tuning” on OpenResearch

About this paper

Yilun Liu, Shimin Tao, Xiaofeng Zhao, Ming Zhu, Wenbing Ma, Junhao Zhu, Chang Su, Yutai Hou and 6 more

Huawei researchers developed CoachLM, an automatic instruction revision system for LLM instruction tuning that enhances data quality while maintaining diversity. This system, trained on expert-revised examples, improved the proportion of high-quality samples in the ALPACA52K dataset from 17.7% to 78.9% and enabled a 7B parameter Alpaca model to outperform 13B Vicuna and achieve up to 21.5% higher win rates against baseline models.

Key insights

• Automatically revising low-quality instruction data maintains dataset diversity while significantly improving quality, a paradigm distinct from filtering-based methods.
• Training the 'coach' LLM on a selected subset of high-impact human revisions (optimal at 30% of human-revised samples) is more efficient and effective than using all available human revisions.
• Higher quality instruction data directly translates to superior instruction-following capabilities in fine-tuned LLMs, even enabling smaller models to outperform larger, less optimally tuned counterparts.

Summary by alphaXiv.