Reproduce “ATLAS-Finance: Benchmarking AI Agents in Realistic Banking Environments” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Elaine Lau, Antoine Pangas, Alexandre Finot, Collin Schweiker, Amir Bouzidi, Hui Wen Goh, Andrew Peter Martin, Jonas Mueller

ATLAS-Finance evaluates agents on 100 end-to-end banking tasks across 13 simulated firms, requiring them to recover instructions and inputs from workplace tools, build Excel workbooks, and submit them correctly. In tests of 11 models, Claude Opus 5 achieved the highest Pass@1 at 10.7%. A controlled comparison found lower rubric scores in workplace conditions; results do not establish live-bank reliability.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.