Reproduce “Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qianjia Cheng, Yuchen Zhang, Zhilin Wang, Yuxin Zuo, Shunkai Zhang, Yuchen Fan, Yu Qiao, Bowen Zhou and 4 more
Researchers from Shanghai AI Laboratory and collaborating universities developed TRICE, a full-pipeline methodology that instills strong thinking models with robust tool-integrated reasoning capabilities, leading to state-of-the-art performance on mathematical benchmarks and cross-domain generalization.
Key insights
- Supervised fine-tuning (SFT) for tool-integrated reasoning progresses through 'form-substance-noise' stages, where selecting the optimal 'substance' stage checkpoint is crucial for stable and effective downstream reinforcement learning.
- Effective teacher-generated tool-use patterns are lightweight, high-frequency, and incrementally build upon a persistent sandbox state, facilitating better student model learning.
- The code executor serves diverse cognitive functions beyond simple computation offloading, frequently assisting in empirical discovery, algorithmic search, and conjecture verification.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.