Reproduce “Evaluating Byte and Wordpiece Level Models for Massively Multilingual Semantic Parsing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Massimo Nicosia, Francesco Piccinno
A study from Google Research compared byte-level (ByT5) and wordpiece-based (mT5) models for multilingual semantic parsing across 51 languages, demonstrating ByT5's stronger performance at smaller model sizes and the effectiveness of synthetic data generation for data-scarce languages. Models performed less accurately on localized test examples compared to translated ones, highlighting the need for culturally appropriate data.
Key insights
- ByT5 models exhibit stronger Exact Match accuracy than mT5 at smaller model sizes, attributed to more efficient parameter utilization.
- Synthetic data generated by the Translate-and-Fill (TAF) method is effective, yielding performance within 5% of models trained on full gold data.
- Cross-lingual transfer analysis shows better performance within language families and across similar writing systems, and English surprisingly acts as a good receiver language for transfer.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.