Reproduce “Reliable Evaluation of AI Agents for Investment Research: A Framework for Performance and Validity Assessment” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Dongnuan Tian, Yan Yu, Yangfan Liu, Wenjun Tang, Junting Kan, Xiang Wang, Sicong Liu, Yangyang Zhang and 1 more

The authors present a framework for evaluating financial research agents by separating report quality, execution quality, and evaluation validity. It verifies inputs and evidence, then matches grading to criteria. Across three agents and 62 tasks, Codex’s advantage over OpenClaw rises from 7.20% to 8.19% without execution efficiency; the results do not guarantee correct judgments or generalize beyond this testbed.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.