Reproduce “Large Language Models as Annotators for Machine Translation Quality Estimation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sidi Wang, Sophie Arnoult, Amir Kamran
The authors train reference-free COMET models to estimate machine-translation quality from GPT-4o’s error annotations. They ask GPT-4o to mark translation errors and severity, then turn those labels into training targets. Across Chinese–English and English–German, the models are competitive overall with models trained on human MQM labels, and correlate better in some low-quality buckets, but not across every bucket or metric.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.