Reproduce “FinMMR: Make Financial Numerical Reasoning More Multimodal, Comprehensive, and Challenging” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zichen Tang, Haihong E, Jiacheng Liu, Zhongjun Yang, Rongjin Li, Zihua Rong, Haoyang He, Zhuodi Hao and 13 more

FinMMR benchmarks multimodal models on financial questions requiring image reading, financial reasoning, and precise calculation. Questions use visual data and may include irrelevant images. In a zero-shot test of 15 multimodal large language models, Claude 3.7 Sonnet with 64K extended thinking scored 53.00% on the Hard test subset using IO prompting, within a strict 0.2% error tolerance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.