Reproduce “DocFinQA: A Long-Context Financial Reasoning Dataset” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Varshini Reddy, Rik Koncel-Kedziorski, Viet Dac Lai, Michael Krumdick, Charles Lovering, Chris Tanner

DocFinQA extends financial question answering to full SEC filings, pairing 7,437 questions with executable Python programs. Unlike FinQA’s selected excerpts, its documents average 123,453 words, compared with 687 for FinQA. In a 200-document subset of filings at least 100K tokens, GPT-4 with retrieval scored 47.5%, versus 23% with retrieval-free System 2 Attention; this result is limited to that case study.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.