Reproduce “Post-Training Science for Supervised Fine-Tuning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Charles O'Neill, Mudith Jayasekara, Harry Partridge

This research from Baseten empirically measured the impact of various supervised fine-tuning (SFT) hyperparameters on large language models using real-world datasets, establishing data-driven guidelines for adaptation. The study identified optimal LoRA learning rates and ranks, quantified LoRA's efficiency relative to full fine-tuning, and demonstrated that geometry-aware optimizers like Muon can improve generalization and achieve flatter loss minima while maintaining task performance.

Key insights

  • Optimal LoRA learning rates (10^-3) are significantly higher and more consistent across models and datasets than full fine-tuning rates.
  • LoRA achieves nearly all (median 98%) of full fine-tuning's performance gains with substantially fewer trainable parameters (3.1-12.6%), offering significant resource efficiency.
  • Validation NLL reliably predicts downstream task quality within a fixed recipe but does not consistently transfer across model families, and loss flatness correlates with retained general capabilities, not necessarily task performance at matched loss.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.