Reproduce “Lost in the Maze: Overcoming Context Limitations in Long-Horizon Agentic Search” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Howard Yen, Yoonsang Lee, Ashwin Paranjape, Mengzhou Xia, Thejas Venkatesh, Jack Hessel, Danqi Chen, Yuhao Zhang

The research introduces SLIM (Simple Lightweight Information Management), a framework designed for long-horizon agentic search that enhances efficiency by selectively managing context and tool interactions. It achieves up to 26 absolute points higher performance on tasks like GAIA and reduces tool calls by 4-6x compared to other open-source frameworks, while also significantly lowering hallucination rates.

Key insights

  • Poor context management, rather than solely LLM reasoning, is a primary bottleneck for existing long-horizon agentic search systems, leading to early termination and resource exhaustion.
  • Decoupling search result presentation from deep content browsing, and selectively retrieving information, drastically improves efficiency and scalability for complex information-seeking tasks.
  • Systematic trajectory analysis using an error taxonomy reveals specific failure patterns, highlighting a trade-off between information discovery (improved by SLIM) and an agent's ability to synthesize a correct answer from retrieved content.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.