Reproduce “A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuemei Xu, Ling Hu, Jiayi Zhao, Zihan Qiu, Kexin XU, Yuqi Ye, Hanwen Gu
A survey paper from Beijing Foreign Studies University offers a comprehensive overview of Multilingual Large Language Models (MLLMs), examining their evolution, key technical underpinnings, and persistent challenges related to training corpora, representation alignment, and inherent biases. The work highlights the overwhelming English dominance in training data (often over 90%), persistent language misalignment issues, and the scarcity of robust multilingual bias evaluation benchmarks.
Key insights
- MLLMs exhibit a "curse of multilinguality," showing unsatisfactory performance on low-resource languages due to data scarcity and limited overlap with high-resource languages.
- Training corpora for MLLMs overwhelmingly favor English and a few Indo-European languages, leading to severe underrepresentation of many global language families.
- Achieving truly universal language representation alignment remains challenging, with performance influenced by typological distance and specific pre-training configurations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.