Reproduce “TInR: Exploring Tool-Internalized Reasoning in Large Language Models” on OpenResearch

About this paper

Qiancheng Xu, Yongqi Li, Fan Liu, Hongru Wang, Min Yang, Wenjie Li

TInR-U is a framework that enables Large Language Models (LLMs) to perform Tool-Internalized Reasoning by embedding tool knowledge directly into their parameters. This approach improves tool mastery and scalability, while maintaining constant inference efficiency as the toolset size increases, outperforming traditional tool-integrated methods across diverse benchmarks.

Key insights

• Directly internalizing tool knowledge into LLM parameters, rather than relying on external documentation, provides a more efficient and scalable paradigm for tool-augmented reasoning.
• Bidirectional knowledge alignment, which includes memorization of tool documentation, recall of documentation from tool tokens, and grounding in practical usage scenarios, is crucial for robust tool knowledge acquisition.
• A structured training approach that combines SFT for initial behavior alignment and RL for robust reasoning, alongside a two-step tool action generation, significantly enhances tool-reasoning coordination and reduces model burden.

Summary by alphaXiv.