Reproduce “Fast Vocabulary Transfer for Language Model Compression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Leonidas Gee, Andrea Zugarini, Leonardo Rigutini, Paolo Torroni
Researchers introduce Fast Vocabulary Transfer (FVT), a method to adapt a language model's vocabulary to a specific domain for compression. FVT achieves up to 55% model size reduction and 2.75x inference speedup with minimal impact on F1 scores when combined with Knowledge Distillation.
Key insights
- Intelligent initialization of new vocabulary embeddings, via subword averaging, substantially improves performance compared to random initialization.
- Model performance does not always degrade monotonically with vocabulary size reduction; optimal vocabulary size can be smaller than the original and varies by domain.
- The effectiveness of vocabulary transfer is domain-dependent, providing greater efficiency gains in highly specialized domains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.