Reproduce “Learning from Synthetic Data Improves Multi-hop Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Anmol Kabra, Yilun Yin, Albert Gong, Kamilė Stankevičiūtė, Dongyoung Go, Johann Lee, Katie Z. Luo, Carla P. Gomes and 1 more
Reinforcement Learning fine-tuning on rule-generated synthetic data significantly enhances multi-hop reasoning in Large Language Models. Models trained on fictional, semantically simple data acquire a transferable ability to compose knowledge, resulting in relative F1 score improvements ranging from 56% to 131% on real-world benchmarks.
Key insights
- LLMs can learn "knowledge composition," a generalizable and domain-independent reasoning skill, even from semantically simple, fictional synthetic data.
- Rule-generated synthetic data presents a scalable, free, and verifiable resource for continuously improving LLM reasoning capabilities.
- Reinforcement Learning fine-tuning is more effective than Supervised Fine-Tuning in imparting transferable reasoning skills from synthetic data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.