Reproduce “Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hao Zhao, Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion
EPFL researchers present a robust baseline for instruction fine-tuning, demonstrating that fine-tuning large language models on just 1,000 instruction-response pairs with the longest answers outperforms models trained on much larger or meticulously curated datasets. This simple approach, optionally enhanced by lightweight LLM-based refinement, enables models like Llama-2-7B to achieve highly competitive performance on standard alignment benchmarks such as AlpacaEval 2.0.
Key insights
- A simple length-based heuristic, selecting the 1,000 longest responses from large instruction datasets, serves as an exceptionally effective and cost-efficient proxy for high-quality instruction fine-tuning data, challenging the notion that complex data curation is essential.
- The observed performance gains are not merely due to LLM-as-a-judge bias towards longer responses; controlled experiments and human evaluations confirm genuinely higher quality in models trained on longer examples.
- Training on longer responses leads to slower convergence of training loss, suggesting these examples are 'harder to fit' and provide richer supervisory signals that enhance model generalization and even improve performance on factual knowledge benchmarks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.