Reproduce “Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lei Liu, Jimmy Xiangji Huang
Researchers at York University developed a prompt learning strategy to mitigate catastrophic forgetting in multilingual pre-trained language models, enabling improved open-domain dialogue generation in few-shot and multitask cross-lingual settings. The method enhanced response fluency and diversity across six non-English languages, as evidenced by automatic and human evaluations.
Key insights
- Catastrophic forgetting is indeed prevalent in few-shot cross-lingual transfer (FS-XLT) and multitask learning (MTL) for open-domain dialogue generation, leading to English or ungrammatical responses in target languages.
- Realigning the fine-tuning task with the pre-training objectives through prompt learning can effectively preserve the inherent multilinguality of mPLMs.
- The specific prompt formulation, inspired by mT5's span corruption, successfully activates the mPLM's multilingual knowledge during fine-tuning, allowing it to generate appropriate target language responses.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.