Reproduce “Generating Adversarial Texts for Machine Translation via GRPO” on OpenResearch

About this paper

Florian Zogaj, Jakob Hütteneder, Giovanni De Muri, Federico Villa, Aryan Sood, Vilém Zouhar

Researchers at ETH Zurich developed a reinforcement learning framework using Group Relative Policy Optimization (GRPO) to fine-tune a large language model, generating adversarial texts that significantly reduce machine translation quality. The method decreased average COMET scores by 0.15 on WMT25 and 0.20 on WMT24, while largely preserving the linguistic quality and meaning of the original sentences.

Key insights

• Explicit reinforcement learning is essential for generating challenging rewrites, as an instructed base LLM primarily produces natural paraphrases with minimal impact on translation difficulty.
• Averaging translation difficulty reward signals from multiple MT systems (e.g., COMET-Average) yields a more robust and generalizable fine-tuned model.
• The generated adversarial texts achieve substantial translation quality reduction while largely preserving linguistic quality and semantic meaning, indicating the challenge stems from subtle linguistic complexities rather than overt errors.

Summary by alphaXiv.