Reproduce “Sequence Level Training with Recurrent Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Marc'Aurelio Ranzato, Sumit Chopra, Michael Auli, Wojciech Zaremba
The paper from Facebook AI Research presents MIXER, a novel training algorithm for recurrent neural networks that addresses exposure bias and loss mismatch in text generation. It integrates word-level cross-entropy with sequence-level REINFORCE using an incremental learning strategy, achieving higher sequence quality and faster inference compared to standard methods across multiple tasks.
Key insights
- Directly optimizing sequence-level, non-differentiable metrics through REINFORCE, when properly initialized, significantly improves text generation quality beyond word-level cross-entropy.
- Mitigating exposure bias by gradually transitioning from ground truth to model-generated inputs during training is crucial for robust performance.
- Incremental learning (warm-starting with cross-entropy and annealing to full REINFORCE) makes applying reinforcement learning practical for large-vocabulary text generation tasks where vanilla REINFORCE often fails.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.