Reproduce “Towards Effective Code-Integrated Reasoning” on OpenResearch

About this paper

Fei Bai, Yingqian Min, Beichen Zhang, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, Zheng Liu, Zhongyuan Wang and 1 more

This paper from Renmin University, DataCanvas, and BAAI presents a systematic approach to improve the training effectiveness and stability of tool-augmented reinforcement learning for code-integrated reasoning in large language models. The method achieves state-of-the-art performance on mathematical reasoning benchmarks and provides mechanistic insights into how code integration extends model capabilities, offering efficiency over traditional reasoning methods.

Key insights

• Code-integrated reasoning empirically expands the fundamental capability boundaries of LLMs, enabling them to solve problems beyond their intrinsic knowledge.
• This approach achieves comparable accuracy to powerful Long-CoT models but with significantly shorter response lengths (less than 15-20% of tokens), indicating higher efficiency.
• Non-executable code can paradoxically be beneficial, as error feedback prompts the model to reflect and revise its approach, leading to correct solutions; conversely, executable but logically flawed code can mislead the model.

Summary by alphaXiv.