Abstract Machine translation (MT) is an important sub-field of natural language processing
that aims to translate natural languages using computers. In recent years, end-to-end neural …

Multi-headed attention is a driving force behind recent state-of-the-art NLP models. By
applying multiple attention mechanisms in parallel, it can express sophisticated functions …

Most machine translation systems generate text autoregressively from left to right. We,
instead, use a masked language modeling objective to train a model to predict any subset of …

This paper presents two strong methods, CTC and Imputer, for non-autoregressive machine
translation that model latent alignments with dynamic programming. We revisit CTC for …

While most neural machine translation (NMT) systems are still trained using maximum
likelihood estimation, recent work has demonstrated that optimizing systems to directly …

Unsupervised machine translation (UMT) has recently attracted more attention from
researchers, enabling models to translate when languages lack parallel corpora. However …

We explore the application of very deep Transformer models for Neural Machine Translation
(NMT). Using a simple yet effective initialization technique that stabilizes training, we show …

Non-autoregressive machine translation models significantly speed up decoding by
allowing for parallel prediction of the entire target sequence. However, modeling word order …

Transformer-based models have brought a radical change to neural machine translation. A
key feature of the Transformer architecture is the so-called multi-head attention mechanism …

While diffusion models have achieved great success in generating continuous signals such
as images and audio, it remains elusive for diffusion models in learning discrete sequence …