Reproduce “SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Fangyuan Xu, Rujun Han, Yanfei Chen, Zifeng Wang, I-Hung Hsu, Jun Yan, Vishy Tirumalashetty, Eunsol Choi and 2 more

SAGE is an agentic data generation framework, developed by Google Cloud AI Research and NYU, that employs a dual-agent system with execution feedback to automatically produce high-quality, difficulty-controlled question-answer pairs for deep search. Training deep search agents with SAGE-generated data results in up to a 29% relative performance improvement on in-domain benchmarks and demonstrates effective generalization to real-world open-web search environments.

Key insights

  • Execution feedback from a search agent is crucial for generating high-quality synthetic deep search data, effectively aligning the intended difficulty with the agent's actual performance.
  • The SAGE framework produces data that encourages a broader and more diverse range of complex reasoning strategies in deep search agents, including conflict resolution and temporal reasoning.
  • Deep search agents trained on fixed-corpus data can effectively generalize to real-world open-web search environments, enabling a more cost-efficient training and deployment strategy.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.