Reproduce “On the Cross-lingual Transferability of Monolingual Representations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mikel Artetxe, Sebastian Ruder, Dani Yogatama
Researchers from the University of the Basque Country (UPV/EHU) and DeepMind developed MONOTRANS, a method showing that deep monolingual models can achieve competitive zero-shot cross-lingual transfer performance without relying on shared subword vocabularies or joint multilingual training. This work also contributes XQuAD, a professionally translated cross-lingual question answering dataset for more rigorous evaluation.
Key insights
- Deep monolingual models appear to learn robust linguistic abstractions that are inherently generalizable across languages, challenging the necessity of shared vocabularies and joint training.
- The effective number of subword tokens allocated per language in a vocabulary is more critical for cross-lingual performance than the mere presence of shared vocabulary items.
- Semantic abstractions transfer more readily across languages than syntactic ones when using monolingual models and lexical alignment methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.