Reproduce “Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Polina Tsvilodub, Max Höth, Michael Franke, Björn Deiseroth, Carina Kauf
The authors introduce SURE, a GRPO fine-tuning reward that combines task outcome with a penalty for reasoning after identifying missing task-critical information. Across four fine-tuned model families, SURE increased abstention recall by an average of 0.175 relative to initial models and produced 44% shorter chains of thought. Answerable-question accuracy was 0.03 lower, but not every metric improved.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.