Reproduce “Control-R: Towards controllable test-time scaling” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Di Zhang, Weida Wang, Junxian Li, Xunzhi Wang, Jiatong Li, Jianbo Wu, Jingdi Lei, Haonan He and 5 more

The Control-R framework enables dynamic, test-time adjustment of reasoning depth and breadth in Large Language Models for complex tasks, using 11 distinct Reasoning Control Fields to mitigate both insufficient and redundant reasoning. A 32B model fine-tuned with this approach achieved 70.0% on AIME2024 and 93.2% on MATH500, demonstrating that explicit control over reasoning processes can enhance performance and efficiency.

Key insights

  • Long Chain-of-Thought reasoning can be effectively modeled as an "In-Context Tree Search" process, allowing for structured control over exploration.
  • Reasoning Control Fields (RCFs) are essential, not merely auxiliary, for achieving high performance in complex reasoning tasks, as demonstrated by significant performance drops in their absence.
  • Controllable test-time scaling allows for a dynamic balance between computational efficiency and reasoning thoroughness, adapting to varying task requirements.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.