Reproduce “Question Translation Training for Better Multilingual Reasoning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Wenhao Zhu, Shujian Huang, Fei Yuan, Shuaijie She, Jiajun Chen, Alexandra Birch

Researchers at Nanjing University, Shanghai AI Lab, and the University of Edinburgh developed the Question Alignment (QAlign) framework to enhance multilingual mathematical reasoning in open-source LLMs. This two-stage fine-tuning process implicitly aligns non-English questions with their English counterparts, leading to an average accuracy improvement of 11.2% (LLaMA2-7B) and 13.2% (LLaMA2-13B) on the MGSM benchmark compared to English-only baselines.

Key insights

  • Focusing language alignment specifically on translating non-English questions to English (X→En) is more effective than translating entire Chain-of-Thought responses, avoiding quality degradation.
  • A dedicated two-stage training order, first for question alignment and then for response alignment, is critical; reversing the order or combining stages leads to performance degradation.
  • The methodology enhances consistency in predicted answers for the same question across different languages, suggesting robust transfer of reasoning ability.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.