Reproduce “FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhoujun Cheng, Haoyu Dong, Ran Jia, Pengfei Wu, Shi Han, Fan Cheng, Dongmei Zhang
FORTAP (FORmula-driven TAble Pretraining) introduces a pretraining framework that leverages real-world spreadsheet formulas to instill numerical reasoning capabilities in table understanding models. This approach establishes new state-of-the-art results on formula prediction, table question answering, and cell type classification benchmarks.
Key insights
- Real-world spreadsheet formulas provide an abundant, authentic, and scalable supervision signal for teaching numerical reasoning to table models.
- Explicitly training models to predict numerical references and calculations, derived directly from formula structures, significantly enhances their understanding of tabular data relationships.
- The combination of tasks focusing on cell referencing, operation prediction, and formula syntax encoding leads to a more comprehensive and robust numerical reasoning capability across diverse table types.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.