Can Automatic Metrics Assess High-Quality Translations?

S Agrawal, A Farinhas, R Rei, AFT Martins - arxiv preprint arxiv …, 2024 - arxiv.org
Automatic metrics for evaluating translation quality are typically validated by measuring how
well they correlate with human assessments. However, correlation methods tend to capture …

A survey of context in neural machine translation and its evaluation

S Castilho, R Knowles - Natural Language Processing, 2024 - cambridge.org
The question of context in neural machine translation often focuses on topics related to
document-level translation or intersentential context. However, there is a wide range of other …

A Survey of AI Reliance

S Eckhardt, N Kühl, M Dolata, G Schwabe - arxiv preprint arxiv …, 2024 - arxiv.org
Artificial intelligence (AI) systems have become an indispensable component of modern
technology. However, research on human behavioral responses is lagging behind, ie, the …

A Context-aware Framework for Translation-mediated Conversations

J Pombal, S Agrawal, P Fernandes, E Zaranis… - arxiv preprint arxiv …, 2024 - arxiv.org
Effective communication is fundamental to any interaction, yet challenges arise when
participants do not share a common language. Automatic translation systems offer a …

Translation in the Hands of Many: Centering Lay Users in Machine Translation Interactions

B Savoldi, A Ramponi, M Negri, L Bentivogli - arxiv preprint arxiv …, 2025 - arxiv.org
Converging societal and technical factors have transformed language technologies into user-
facing applications employed across languages. Machine Translation (MT) has become a …

SpeechQE: Estimating the Quality of Direct Speech Translation

HJ Han, K Duh, M Carpuat - arxiv preprint arxiv:2410.21485, 2024 - arxiv.org
Recent advances in automatic quality estimation for machine translation have exclusively
focused on written language, leaving the speech modality underexplored. In this work, we …

Social and Ethical Risks Posed by General-Purpose LLMs for Settling Newcomers in Canada

I Nejadgholi, M Molamohammadi… - arxiv preprint arxiv …, 2024 - arxiv.org
The non-profit settlement sector in Canada supports newcomers in achieving successful
integration. This sector faces increasing operational pressures amidst rising immigration …

Segment-level evaluation of machine translation metrics

N Moghe - 2024 - era.ed.ac.uk
Most metrics evaluating Machine Translation (MT) claim their effectiveness by demonstrating
their ability to distinguish the quality of different MT systems over a large corpus (system …

[KNJIGA][B] Designing for Reliability in Algorithmic Systems

S Robertson - 2023 - search.proquest.com
As we introduce complex algorithmic systems into decision-making in high-stakes domains,
system designers need principled approaches to help people set their expectations of these …