Reproduce “Teaching Small Language Models to Reason” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, Aliaksei Severyn
This research successfully transfers the advanced Chain of Thought (CoT) reasoning capabilities from large language models (LLMs) like PaLM 540B to significantly smaller models such as T5, demonstrating substantial improvements in accuracy across arithmetic, commonsense, and symbolic reasoning tasks. The approach makes sophisticated reasoning more accessible and efficient, enabling smaller models to perform comparably to much larger baselines on specific tasks.
Key insights
- Providing the target answer to the teacher LLM during CoT generation is a critical modification, significantly improving the quality of the generated reasoning paths and thereby the effectiveness of distillation.
- CoT knowledge distillation enables small language models to learn the reasoning *process* effectively, with T5 XXL's reasoning accuracy (with external calculator) on GSM8K approaching that of the PaLM 540B teacher.
- The method offers substantial efficiency gains, allowing significantly smaller student models (e.g., T5 Base, 44x smaller) to match or exceed the performance of much larger non-CoT baselines, and achieving improvements with reduced training data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.