Reproduce “PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literature” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Daoyu Wang, Mingyue Cheng, Shuo Yu, Zirui Liu, Ze Guo, Xin Li, Qi Liu

PaperArena evaluates agents on research questions that require finding and interpreting evidence across scientific papers, using tools, and combining results into grounded answers. PaperArena-Hub provides modular tools and supports single- and multi-agent workflows. Gemini 2.5 Pro scores 38.78% overall and 18.47% on hard questions; experts score 83.50% on a separate 200-question subset, so this is not a matched comparison.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.