Reproduce “Semi-supervised Sequence Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Andrew M. Dai, Quoc V. Le
This paper presents a semi-supervised learning framework that leverages vast amounts of unlabeled text data to pre-train Long Short-Term Memory (LSTM) networks. The approach addresses LSTM training instability and significantly improves performance on text classification benchmarks, achieving an IMDB error rate of 7.24% and a 20 Newsgroups error rate of 15.6%.
Key insights
- Unsupervised pre-training significantly improves the training stability and performance of Long Short-Term Memory (LSTM) networks, particularly for long sequences.
- Learning general language representations from large amounts of unlabeled text data through methods like language modeling or sequence autoencoding effectively initializes LSTMs, leading to better generalization on downstream tasks.
- The
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.