Reproduce “Breadcrumbs Reasoning: Memory-Efficient Reasoning with Compression Beacons” on OpenResearch

About this paper

Giovanni Monea, Yair Feldman, Shankar Padmanabhan, Kianté Brantley, Yoav Artzi

A memory-efficient reasoning mechanism called Breadcrumbs Reasoning (BR) was developed by researchers at Cornell University, employing periodic KV cache compression into 'beacon tokens' via a joint reinforcement learning-distillation framework. This method enables large language models to generate substantially longer reasoning chains, achieving up to 219.7% higher Area Under Accuracy-Cache compared to uncompressed models on complex tasks, with comparable inference times.

Key insights

• The value of past reasoning steps often diminishes, suggesting that not all historical information needs to be explicitly stored, enabling compression.
• Learned compression is more effective than training-free heuristic methods for complex, multi-step reasoning tasks.
• A single model can be effectively trained to support multiple compression ratios, providing flexible memory management at inference time.
• Compressing information, particularly numerical and algebraic data for tasks like LinSys, poses specific challenges, sometimes leading to increased arithmetic errors.

Summary by alphaXiv.