Reproduce “CoLT: Reasoning with Chain of Latent Tool Calls” on OpenResearch

About this paper

Fangwei Zhu, Zhifang Sui

CoLT introduces a framework that enables Large Language Models to perform efficient multi-step reasoning by compressing explicit reasoning steps into latent parametric tool calls, which are then unpacked by differentiable external decoders. This approach achieves higher accuracy and reduced reasoning chain length on mathematical reasoning benchmarks while maintaining interpretability with minimal training.

Key insights

• Complex multi-token reasoning steps can be effectively compressed into a few latent seed tokens, which specialized external decoders can accurately expand back into explicit text.
• Implementing latent reasoning as differentiable 'parametric tool calls' allows for flexible integration and joint optimization with the main LLM, preserving its pre-trained abilities with minimal intrusion.
• Reinforcement learning can further enhance reasoning performance and efficiency by guiding the model to discover concise and effective reasoning trajectories beyond those in supervised datasets.

Summary by alphaXiv.