Reproduce “Augmenting Text to Increase Translation Difficulty” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

William Kalikman, Šimon Sukup, Michal Tešnar, Vilém Zouhar

Researchers at ETH Zurich developed Adversarial Translation Optimization (ATO), a gradient-based framework that augments source texts to significantly increase machine translation difficulty while attempting to preserve linguistic coherence. ATO-TwoPhase, a variant, lowered xCOMET scores of translations from 0.93 to 0.82 and MetricX scores from 0.91 to 0.82 across various MT models, while maintaining perplexity comparable to original texts.

Key insights

  • Gradient-based adversarial optimization, when guided by differentiable difficulty estimators, can systematically generate texts that are harder for MT systems to translate.
  • Integrating explicit differentiable fluency constraints or a dedicated fluency recovery phase is crucial for balancing increased difficulty with preserving linguistic well-formedness.
  • A two-phase optimization approach can achieve both high translation difficulty and surprising fluency (as measured by perplexity) by decoupling the search for difficulty from the refinement of linguistic naturalness.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.