Reproduce “Tandem Training for Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Robert West, Ashton Anderson, Ece Kamar, Eric Horvitz
The authors introduce tandem training, in which a senior model and a frozen weaker junior take turns writing a shared solution, with the senior updated after successful joint rollouts. On GSM8K, the specialist’s benchmark-specific notation fell from 99% to 0% within 20 updates, while accuracy was 33% versus the junior’s 24%. These results concern Llama-2 model partners, not human understanding.
Key insights
- Intelligibility in powerful AI models can be implicitly encouraged through a collaborative training setup, without requiring explicit intelligibility labels or human judges.
- Randomized "handoffs" to a weaker model during training act as a regularization technique, preventing the senior model from developing overly complex or idiosyncratic reasoning patterns.
- This training paradigm enables advanced models to adapt their communication style, including language choice and jargon usage, to a less capable partner while maintaining solution correctness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.