Reproduce “AgenticFinBench: Benchmarking Financial Agents on Diverse Tasks from Real-World Requests” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shu Zhou, Ziyang He, Yuxin Rao, Yudong Fang, Jiali Qian, Dongyuan Chen, Juntao Zhang, Yong Hu
AgenticFinBench evaluates financial agents with a question-specific rubric and a claim-level audit. The audit checks whether claims are supported by the agent’s trace and whether external evidence verifies them, then incorporates confirmed factual errors into quality scores. On the same 230 questions, DeepSeek V4.1 Flash on Pi fell from first to fourth, while GPT-5.6 Sol on Codex CLI rose from fifth to first; these are point-estimate ranks, not necessarily statistically different.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.