Recent work in natural language processing (NLP) has yielded appealing results from
scaling model parameters and training data; however, using only scale to improve …

Autoregressive decoding limits the efficiency of transformers for Machine Translation (MT).
The community proposed specific network architectures and learning-based methods to …

Much recent effort has been invested in non-autoregressive neural machine translation,
which appears to be an efficient alternative to state-of-the-art autoregressive machine …

NLP research is impeded by a lack of resources and awareness of the challenges presented
by underrepresented languages and dialects. Focusing on the languages spoken in …

Fully non-autoregressive neural machine translation (NAT) is proposed to simultaneously
predict tokens with single forward of neural networks, which significantly reduces the …

Abstract Named Entity Recognition (NER) for low-resource languages is a both practical and
challenging research problem. This paper addresses zero-shot transfer for cross-lingual …

Adversaries may look to steal or attack black-box NLP systems, either for financial gain or to
exploit model errors. One setting of particular interest is machine translation (MT), where …

The attention mechanism is the crucial component of the transformer architecture. Recent
research shows that most attention heads are not confident in their decisions and can be …

Large language models have become increasingly difficult to train because of the growing
computation time and cost. In this work, we present SRU++, a highly-efficient architecture …

Transformers have outperformed recurrent neural networks (RNNs) in natural language
generation. But this comes with a significant computational cost, as the attention …