Reproduce “ATLAS-Finance: Benchmarking AI Agents in Realistic Banking Environments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Elaine Lau, Antoine Pangas, Alexandre Finot, Collin Schweiker, Amir Bouzidi, Hui Wen Goh, Andrew Peter Martin, Jonas Mueller
ATLAS-Finance evaluates agents on 100 end-to-end banking tasks across 13 simulated firms, requiring them to recover instructions and inputs from workplace tools, build Excel workbooks, and submit them correctly. In tests of 11 models, Claude Opus 5 achieved the highest Pass@1 at 10.7%. A controlled comparison found lower rubric scores in workplace conditions; results do not establish live-bank reliability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.