Reproduce “Disentangling Uncertainty in Machine Translation Evaluation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chrysoula Zerva, Taisiya Glushkova, Ricardo Rei, André F. T. Martins
The authors train uncertainty predictors for COMET scores to indicate why a score may be unreliable. HTS and KL target data uncertainty, while DUP predicts score errors. On WMT20 direct-assessment data, HTS achieves a UPS of 0.177 versus 0.134 for deep ensembles. UPS tracks errors, not score accuracy; results do not establish source separation or adaptation to new domains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.