Reproduce “A Survey on Post-training of Large Language Models” on OpenResearch

About this paper

Guiyao Tie, Zeli Zhao, Dingjie Song, Fuyang Wei, Rong Zhou, Yurou Dai, Wen Yin, Zhejian Yang and 18 more

This survey offers the first comprehensive review of Post-training Language Models (PoLMs), systematically classifying methods, datasets, and applications within a novel intellectual framework. It traces the evolution of LLMs across five core paradigms—Fine-tuning, Alignment, Reasoning, Efficiency, and Integration & Adaptation—and identifies critical future research directions.

Key insights

• Post-training is indispensable for transforming generalized pre-trained LLMs into specialized, task-adaptive systems capable of overcoming inherent limitations in reasoning, ethics, and domain-specific performance.
• Reinforcement Learning (RL), in various forms (RLHF, RLAIF, DPO, GRPO), is a cornerstone of modern PoLMs, crucial for achieving robust human preference alignment and significantly advancing complex multi-step reasoning capabilities.
• The quality, diversity, and strategic use of datasets (human-labeled, distilled, synthetic) are paramount for the success of any post-training endeavor, directly impacting model performance and adaptability.

Summary by alphaXiv.