This article presents the most up-to-date, influential automated, semiautomated and human
metrics used to evaluate the quality of machine translation (MT) output and provides the …

The fast advancement in machine translation models necessitates the development of
accurate evaluation metrics that would allow researchers to track the progress in text …

The majority of NLG evaluation relies on automatic metrics, such as BLEU. In this paper, we
motivate the need for novel, system-and data-independent automatic evaluation methods …

We frame the task of machine translation evaluation as one of scoring machine translation
output with a sequence-to-sequence paraphraser, conditioned on a human reference. We …

This paper presents the results of the WMT19 Metrics Shared Task. Participants were asked
to score the outputs of the translations systems competing in the WMT19 News Translation …

To facilitate effective translation modeling and translation studies, one of the crucial
questions to address is how to assess translation quality. From the perspectives of accuracy …

Measuring the performance of natural language processing models is challenging.
Traditionally used metrics, such as BLEU and ROUGE, originally devised for machine …

This work extends and evaluates a two-dimensional automatic evaluation metric for machine
translation, which is designed to operate at the sentence level. The metric is based on the …

Comparing model performances on benchmark datasets is an integral part of measuring
and driving progress in artificial intelligence. A model's performance on a benchmark …

Machine translation (MT) outputs are widely scored using automatic evaluation metrics and
human evaluation scores. The automatic evaluation metrics are expected to be easily …