Assessing the Role of Context in Chat Translation Evaluation: Is Context Helpful and Under What Conditions?

S Agrawal, A Farajian, P Fernandes, R Rei… - Transactions of the …, 2024 - direct.mit.edu
Despite the recent success of automatic metrics for assessing translation quality, their
application in evaluating the quality of machine-translated chats has been limited. Unlike …

Watching the watchers: Exposing gender disparities in machine translation quality estimation

E Zaranis, G Attanasio, S Agrawal… - ar** Data Augmentation for Machine Translation Quality Estimation
S Li, X Bi, T Liu, Z Chen - IEEE/ACM Transactions on Audio …, 2024 - ieeexplore.ieee.org
Machine translation quality estimation (QE) refers to the quality assessment of machine
translations without a given reference translation. Supervised QE models based on neural …

Sentence-Level or Token-Level? A Comprehensive Study on Knowledge Distillation

J Wei, L Sun, Y Leng, X Tan, B Yu, R Guo - arxiv preprint arxiv …, 2024 - arxiv.org
Knowledge distillation, transferring knowledge from a teacher model to a student model, has
emerged as a powerful technique in neural machine translation for compressing models or …

NEO-BENCH: Evaluating Robustness of Large Language Models with Neologisms

J Zheng, A Ritter, W Xu - arxiv preprint arxiv:2402.12261, 2024 - arxiv.org
The performance of Large Language Models (LLMs) degrades from the temporal drift
between data used for model training and newer text seen during inference. One …