Reproduce “Self-Compression of Chain-of-Thought via Multi-Agent Reinforcement Learning” on OpenResearch

About this paper

Yiqun Chen, Jinyuan Feng, Wei Yang, Meizhi Zhong, Zhengliang Shi, Rui Li, Xiaochi Wei, Yan Gao and 4 more

This research introduces Self-Compression via Multi-Agent Reinforcement Learning (SCMA), a framework for compressing Chain-of-Thought reasoning in Large Reasoning Models. It leverages a multi-agent system to identify and prune redundant reasoning steps, achieving accuracy improvements of 4.33% to 10.02% and length reductions of 11.1% to 39.0% across various benchmarks without adding inference overhead.

Key insights

• Multi-agent collaboration with shared parameters enables fine-grained control over reasoning structure, leading to more accurate and efficient CoT compression than single-agent or decoupled approaches.
• Dynamically weighting length penalties based on the perceived importance of reasoning chunks, as assessed by a specialized Scoring Agent, prevents training collapse and protects essential logical structures from over-compression.
• The synergistic co-optimization of specialized agents fosters emergent behaviors like semantic concentration, redundancy pruning, and content-adaptive fine-grained segmentation within reasoning paths.

Summary by alphaXiv.