Neural machine translation: A review
The field of machine translation (MT), the automatic translation of written text from one
natural language into another, has experienced a major paradigm shift in recent years …
natural language into another, has experienced a major paradigm shift in recent years …
Position information in transformers: An overview
Transformers are arguably the main workhorse in recent natural language processing
research. By definition, a Transformer is invariant with respect to reordering of the input …
research. By definition, a Transformer is invariant with respect to reordering of the input …
Language model tokenizers introduce unfairness between languages
Recent language models have shown impressive multilingual performance, even when not
explicitly trained for it. Despite this, there are concerns about the quality of their outputs …
explicitly trained for it. Despite this, there are concerns about the quality of their outputs …
Byt5: Towards a token-free future with pre-trained byte-to-byte models
Most widely used pre-trained language models operate on sequences of tokens
corresponding to word or subword units. By comparison, token-free models that operate …
corresponding to word or subword units. By comparison, token-free models that operate …
Adapterfusion: Non-destructive task composition for transfer learning
Sequential fine-tuning and multi-task learning are methods aiming to incorporate knowledge
from multiple tasks; however, they suffer from catastrophic forgetting and difficulties in …
from multiple tasks; however, they suffer from catastrophic forgetting and difficulties in …
Adversarial attacks on deep-learning models in natural language processing: A survey
With the development of high computational devices, deep neural networks (DNNs), in
recent years, have gained significant popularity in many Artificial Intelligence (AI) …
recent years, have gained significant popularity in many Artificial Intelligence (AI) …
Canine: Pre-training an Efficient Tokenization-Free Encoder for Language Representation
JH Clark, D Garrette, I Turc, J Wieting - Transactions of the Association …, 2022 - direct.mit.edu
Pipelined NLP systems have largely been superseded by end-to-end neural modeling, yet
nearly all commonly used models still require an explicit tokenization step. While recent …
nearly all commonly used models still require an explicit tokenization step. While recent …
Improving massively multilingual neural machine translation and zero-shot translation
Massively multilingual models for neural machine translation (NMT) are theoretically
attractive, but often underperform bilingual models and deliver poor zero-shot translations. In …
attractive, but often underperform bilingual models and deliver poor zero-shot translations. In …
Massively multilingual neural machine translation in the wild: Findings and challenges
We introduce our efforts towards building a universal neural machine translation (NMT)
system capable of translating between any language pair. We set a milestone towards this …
system capable of translating between any language pair. We set a milestone towards this …
Unsupervised neural machine translation
In spite of the recent success of neural machine translation (NMT) in standard benchmarks,
the lack of large parallel corpora poses a major practical problem for many language pairs …
the lack of large parallel corpora poses a major practical problem for many language pairs …