Reproduce “InfinityMATH: A Scalable Instruction Tuning Dataset in Programmatic Mathematical Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bo-Wen Zhang, Yan Yan, Lin Li, Guang Liu
InfinityMath introduces a scalable instruction tuning dataset for programmatic mathematical reasoning, which addresses the limitations of existing methods by decoupling numerical values from problem logic. This approach allows for efficient, vast data augmentation and yields LLMs that demonstrate enhanced robustness to numerical variations, showing substantial performance improvements across multiple mathematical benchmarks.
Key insights
- Decoupling numerical values from mathematical problems enables the creation of generic, number-independent program templates, allowing for vast data augmentation without repeated LLM calls.
- Training LLMs on data where numbers are abstracted and re-populated significantly improves their robustness to numerical variations and maintains logical consistency in reasoning.
- Integrating 'CoT rationale-like' docstrings within generated programs enhances a model's understanding and reasoning capabilities, even when the docstrings do not directly affect program execution.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.