Reproduce “SeqPO-SiMT: Sequential Policy Optimization for Simultaneous Machine Translation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ting Xu, Zhichao Huang, Jiankai Sun, Shanbo Cheng, Wai Lam

SeqPO-SiMT is a sequential policy optimization framework for simultaneous machine translation (SiMT) that models the task as a multi-step reinforcement learning problem. It addresses limitations of noisy alignment data and single-step RLHF by optimizing both translation quality and latency, yielding average COMET score improvements of 1.3 points in low-latency scenarios.

Key insights

  • Directly simulating the streaming translation process as a multi-step RL problem avoids reliance on noisy partial translation data and captures the inherent sequential nature of SiMT.
  • A carefully designed multi-objective reward function that normalizes and fuses quality and latency metrics, including a latency truncation mechanism, is crucial for effectively balancing these competing objectives.
  • Modeling SiMT as a sequential decision-making process allows an LLM policy to make dynamic choices (translate or wait) based on available context, leading to adaptive quality-latency trade-offs.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.