Reproduce “Self-Error-Instruct: Generalizing from Errors for LLMs Mathematical Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Erxin Yu, Jing Li, Ming Liao, Qi Zhu, Boyang Xue, Minghui Xu, Baojun Wang, Lanqing Hong and 2 more
Self-Error-Instruct (SEI) trains math models by grouping similar errors into types, generating targeted practice, and selecting examples using both correct and incorrect validation cases. On Qwen2.5-Math-7B, it raises the unweighted mean Exact Match across six benchmarks from 32.77% to 57.71%, outperforming two error-driven baselines; this zero-shot result follows training on 1,500 examples and does not isolate clustering’s effect.
Key insights
- Generating training data based on identified error types is more effective than random generation or methods focusing on single error instances.
- Both the rate of fixing bad cases and overall test set performance consistently improve through iterative cycles of error analysis and model refinement.
- Training a model from scratch using the aggregated, selected synthesized datasets yields better performance than iterative fine-tuning, potentially due to better mitigation of overfitting.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.