Reproduce “Beyond Distribution Matching: Selective Margin Preservation for Supervised Fine-Tuning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuxuan Wang, Guangxiang Zhao, Xinyu Tan, Yahui Han, Xinxiang Yin, Zhonghai Wu, Xiangzheng Zhang, Lin Sun and 1 more
SRP-SFT preserves selected preference gaps among alternatives while cross-entropy trains the observed response. It penalizes gaps below reference-scaled bounds at teacher-forced prefixes, rather than matching a local distribution; satisfied gaps incur no further penalty. On one-epoch UltraFeedback fine-tuning of Qwen3-4B-Base, it raises MATH-500 pass@1 from 49.56% to 54.56% versus LP-SFT, but evidence is limited to models up to 4B parameters.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.