Reproduce “Answering the Unanswerable Is to Err Knowingly: Analyzing and Mitigating Abstention Failures in Large Reasoning Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yi Liu, Xiangyu Liu, Zequn Sun, Wei Hu
The method monitors reasoning signals to identify unanswerable math questions and guides large reasoning models to abstain rather than guess. It uses a linear probe on hidden states, an instructional prompt, and early exit. On balanced SUM test questions, R1-Distill-Llama-8B’s correct abstention rate reaches 60.92%, while answer accuracy on answerable questions remains comparable to Vanilla; results cover only the tested math datasets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.