Reproduce “Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models” on OpenResearch

About this paper

Haoran Sun, Lixin Liu, Junjie Li, Fengyu Wang, Baohua Dong, Ran Lin, Ruohui Huang

Alibaba Group researchers developed Conifer, a novel instruction tuning dataset and progressive learning scheme, to improve Large Language Models' ability to follow complex instructions with multiple constraints. Models trained with Conifer, particularly a 7B variant, showed enhanced performance on specific instruction-following benchmarks, sometimes matching or exceeding larger 70B models, while maintaining general alignment.

Key insights

• Decomposing the task of generating complex constrained instructions enables powerful LLMs like GPT-4 to effectively produce high-quality and diverse instruction-following data.
• A progressive learning strategy, combining easy-to-hard instructional progression with explicit internal and external process feedback, is crucial for effectively teaching LLMs to adhere to complex constraints.
• Strategic data curation and tailored learning schemes can yield substantial improvements in specific capabilities for smaller LLMs, allowing them to achieve performance comparable to or surpassing much larger models on targeted benchmarks.

Summary by alphaXiv.