Reproduce “Solving Quantitative Reasoning Problems with Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil and 6 more
Google Research developed Minerva, a large language model based on PaLM, that achieves state-of-the-art performance on college-level quantitative reasoning problems by finetuning on a high-quality mathematical dataset and using a majority voting inference technique. This approach enables the model to solve complex math, science, and engineering problems intrinsically, without relying on external tools like calculators.
Key insights
- Large, high-quality technical datasets containing interwoven natural language and formal mathematical expressions (like LaTeX) are crucial for training language models in quantitative reasoning.
- Scaling language models (e.g., to 540 billion parameters) and employing inference-time techniques such as majority voting across multiple sampled solutions significantly enhance accuracy on complex quantitative tasks.
- Language models can develop intrinsic capabilities for symbolic and numerical manipulation and multi-step reasoning without relying on external tools or calculators.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.