Reproduce “The Shape of the DPO Hyperparameter Landscape: What to Measure, How It Moves with Scale, and When Not to Transfer It” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hannan Mahadik, David Salinas, Jacek Golebiowski, Aaron Klein
The authors map how learning rate and β affect DPO post-training across Qwen3 models, datasets, and evaluation goals. Minimum-validation-loss choices have mean normalized regret of 0.50 for static, 0.62 for chat, and 0.53 overall. Rankings also transfer unreliably across scales and datasets, supporting retuning on the target setup; the dataset comparison tests only Dolci-Instruct-DPO and UltraFeedback.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.