Reproduce “Solving General Arithmetic Word Problems” on OpenResearch

About this paper

Subhro Roy, Dan Roth

Subhro Roy and Dan Roth at the University of Illinois, Urbana Champaign developed an algorithmic approach to solve general arithmetic word problems, handling multi-step problems with all four basic operations without relying on predefined templates or extra annotations. The system achieved 45.2% accuracy on a new multi-step Commoncore dataset, significantly outperforming template-based methods, and demonstrated state-of-the-art performance on existing benchmarks.

Key insights

• Complex multi-step arithmetic problems can be effectively decomposed into a unique set of pairwise Lowest Common Ancestor (LCA) operation predictions through the concept of Monotonic Expression Trees, simplifying the overall learning task.
• Structured linguistic features captured by "quantity schemas" (e.g., associated verbs, units, rate information) are crucial for robustly inferring numerical relationships and operations, enabling generalization beyond superficial textual variations.
• Integrating common-sense world knowledge as inference constraints (e.g., answers should be positive or integral) significantly improves accuracy and robustness by pruning invalid solutions in complex reasoning tasks, yielding over 15% absolute improvement on some datasets.

Summary by alphaXiv.