Reproduce “Meta-Learning for Effective Multi-task and Multilingual Modelling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ishan Tarunesh, Sushil Khyalia, Vishwajeet Kumar, Ganesh Ramakrishnan, Preethi Jyothi
Researchers at IIT Bombay and IBM India Research Lab developed a meta-learning framework to simultaneously improve performance across multiple NLP tasks and languages. They found that deeper semantic tasks benefit more from cross-lingual transfer, while shallower token-centric tasks benefit more from cross-task transfer, leading to improved zero-shot generalization on unseen languages across the XTREME benchmark.
Key insights
- Deeper semantic tasks (e.g., Question Answering, Natural Language Inference) consistently benefit more from cross-lingual transfer (Task-Limited meta-learning).
- Shallower token-centric tasks (e.g., Part-of-Speech tagging, Named Entity Recognition) benefit more from cross-task transfer within a single language (Lang-Limited meta-learning).
- Learned sampling (MultiDDS) can identify information-rich tasks but may not always outperform simpler heuristic sampling, especially for initially over-represented tasks or shallower tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.