Reproduce “From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shuoling Liu, Zhiquan Tan, Kun Yi, Hui Wu, Yihan Li, Jiangpeng Yan, Liyuan Chen, Kai Chen and 1 more

The authors present a framework that models web research as linked evidence paths and tests agents on tracing chains, synthesizing sources, reordering information, and rejecting unsupported premises. In a 296-question bilingual benchmark of 16 systems, Grok Deep Research has the highest average, 19.9%. Adding category tools to tracked search raises GPT-5 from 10.48% to 13.29% on this benchmark, a result limited to this setup.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.