Reproduce “SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiacheng Sang, Mengyuan Li, Sanxing Chen, Yukun Huang, Yu Feng, Bhuwan Dhingra

SearchAtlas turns search-agent trajectories into directed graphs showing how retrieved clues support later queries and answer facts, while marking constraints, failed-search responses, and unsupported information. Deterministic extraction and LLM attribution build graphs whose diagnostics measure answer-path structure, constraint grounding, and prior-knowledge reliance. Across 1,350 trajectories, its score reaches 77.6% accuracy and 74.0% positive F1, indicating association with correctness, not truth.

Key insights

  • Evidence structure is more informative when evaluated at the answer-supporting region: constraints mentioned somewhere in a trajectory are not necessarily constraints that contributed to the answer.
  • Sequential-constraint questions produce deeper, wider, and more exploratory graphs, whereas parallel-constraint questions more often support independent evidence aggregation.
  • Process quality and factual correctness are distinct: coherent evidence flow can still target the wrong entity, while a correct answer can result from inefficient search or an unsupported shortcut.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.