The increasing versatility of language models LMs has given rise to a new class of
benchmarks that comprehensively assess a broad range of capabilities. Such benchmarks …

Human evaluation is increasingly critical for assessing large language models, capturing
linguistic nuances, and reflecting user preferences more accurately than traditional …

The COMET metric has blazed a trail in the machine translation community, given its strong
correlation with human judgements of translation quality. Its success stems from being a …

Selecting an automatic metric that best emulates human annotators is often non-trivial,
because there is no clear definition of" best emulates." A meta-metric is required to compare …

Despite rapid advancements in TTS models, a consistent and robust human evaluation
framework is still lacking. For example, MOS tests fail to differentiate between similar …

Reliable human evaluation is critical to the development of successful natural language
generation models, but achieving it is notoriously difficult. Stability is a crucial requirement …

Low-resource machine translation research often requires building baselines to benchmark
estimates of progress in translation quality. Neural and statistical phrase-based systems are …

Nitika Mathur, Timothy Baldwin, and Trevor Cohn. Towards efficient machine translation
evaluation by modelling annotators. In Proceedings of the Australasian Language …

Human evaluation of machine translation is considered the “gold standard” for evaluation,
but it remains a challenging task for which to define best practices. Recent work has focused …

La traduction automatique, en tant que champ de recherche scientifique et de
développement technique, témoignait–au tournant des années 2020–d'un intérêt croissant …