With the rapid progress in Natural Language Processing (NLP), Pre-trained Language
Models (PLM) such as BERT, BioBERT, and ChatGPT have shown great potential in various …

In this paper, we present the joint contribution of Unbabel and IST to the WMT 2022 Metrics
Shared Task. Our primary submission–dubbed COMET-22–is an ensemble between a …

Widely used learned metrics for machine translation evaluation, such as Comet and Bleurt,
estimate the quality of a translation hypothesis by providing a single sentence-level score …

Generative large language models (LLMs), eg, ChatGPT, have demonstrated remarkable
proficiency across several NLP tasks, such as machine translation, text summarization …

Quality estimation models have been developed to assess the corrections made by
grammatical error correction (GEC) models when the reference or gold-standard corrections …

Automatically evaluating the quality of language generation is critical. Although recent
learned metrics show high correlation with human judgement, these metrics can not explain …

Automatic evaluation of machine translation (MT) is a critical tool driving the rapid iterative
development of MT systems. While considerable progress has been made on estimating a …

Neural sequence generation models are known to “hallucinate”, by producing outputs that
are unrelated to the source text. These hallucinations are potentially harmful, yet it remains …

The increasing versatility of language models (LMs) has given rise to a new class of
benchmarks that comprehensively assess a broad range of capabilities. Such benchmarks …

Abstract Machine Translation Quality Estimation (QE) aims to evaluate the quality of
machine translation without relying on references. Recently, Large-scale Language Model …