Reproduce “Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Qiaoyu Tang, Hao Xiang, Le Yu, Bowen Yu, Yaojie Lu, Xianpei Han, Le Sun, WenJuan Zhang and 6 more

DeepMiner introduces a novel framework for training deep search agents that overcomes turn limits by combining a reverse construction method for complex question generation with an elegant dynamic context window strategy. This approach enables agents to sustain nearly 100-turn interactions within a standard 32k context and achieves state-of-the-art performance among open-source models, outperforming larger proprietary systems on benchmarks like BrowseComp.

Key insights

  • Training with deliberately constructed, complex multi-source reasoning data is crucial for developing advanced cognitive behaviors, outperforming models trained on conventional, simpler datasets.
  • An intelligent dynamic context management strategy, such as the sliding window, can effectively overcome fixed context window limitations without information loss, enabling truly long-horizon interactions.
  • Integrating context management directly into the end-to-end reinforcement learning process, with consistent training-testing conditions, yields superior multi-step reasoning and strategic search capabilities.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.