Reproduce “BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junxiao Yang, Jinzhe Tu, Haoran Liu, Xiaoce Wang, Chujie Zheng, Zhexin Zhang, Shiyao Cui, Caishun Chen and 4 more
BARREL introduces a three-stage training framework for Large Reasoning Models (LRMs) that enhances factual reliability by teaching them to admit ignorance rather than hallucinate answers. The framework improves the reliability of DeepSeek-R1-Distill-Llama-8B from a baseline of 39.33% to 61.48% while maintaining a competitive accuracy of 40.7%.
Key insights
- Pathological reasoning patterns in LRMs are root causes of factual unreliability and can be mitigated through targeted SFT and an explicit focus on boundary awareness.
- Explicitly rewarding truthful rejection (using a medium-level reward for "I don't know") in reinforcement learning is crucial for models to learn to admit ignorance, a critical design element often overlooked.
- Combining SFT with carefully constructed reasoning traces and subsequent GRPO is necessary to effectively instill deliberative, boundary-aware reasoning and robust refusal capabilities in LRMs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.