Reproduce “Rethinking "RL Generalizes, SFT Memorizes": The Role of SFT Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yunlong Hou, Fengzhuo Zhang, Yuan Cheng, Jiachun Pan, Xingyao Li, Zhuoran Yang
The authors test whether supervised fine-tuning (SFT) transfers when given varied solution attempts than reinforcement learning (RL). They train Qwen2.5-7B, matching SFT rollouts to RL and mixing RL-tuned-model and Qwen3-14B examples. The 90/10 mixture scores 44.4 versus RL’s 43.8 on the normalized average of 12 reasoning tasks, a nominal edge in this setup, not a proven seed-level win.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.