Reproduce “Turning Dust into Gold: Distilling Complex Reasoning Capabilities from LLMs by Leveraging Negative Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiwei Li, Peiwen Yuan, Shaoxiong Feng, Boyuan Pan, Bin Sun, Xinglin Wang, Heda Wang, Kan Li
A framework leverages "negative data" – incorrect rationales generated by large language models – to enhance the complex arithmetic reasoning capabilities of smaller language models. This approach achieved an approximate 75.75% increase in accuracy on arithmetic benchmarks and improved out-of-distribution performance.
Key insights
- Incorrect rationales (negative data) generated by LLMs contain valuable, complementary knowledge that can effectively guide a student model to avoid errors and absorb diverse reasoning perspectives.
- Dynamically integrating knowledge from both correct and incorrect rationales, allowing for both positive and negative influences, is more effective than simply mixing data or only using positive feedback.
- Calibrating self-enhancement and adaptively weighting candidate answers during inference using insights derived from negative data leads to more robust and accurate model specialization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.