The field of machine translation (MT), the automatic translation of written text from one
natural language into another, has experienced a major paradigm shift in recent years …

Natural language processing tasks, such as question answering, machine translation,
reading comprehension, and summarization, are typically approached with supervised …

We introduce a new large-scale NLI benchmark dataset, collected via an iterative,
adversarial human-and-model-in-the-loop procedure. We show that training models on this …

A possible explanation for the impressive performance of masked language model (MLM)
pre-training is that such models have learned to represent the syntactic structures prevalent …

Dual encoders perform retrieval by encoding documents and queries into dense low-
dimensional vectors, scoring each document by its inner product with the query. We …

To measure how well pretrained representations encode some linguistic property, it is
common to use accuracy of a probe, ie a classifier trained to predict the property from the …

Although backdoor learning is an active research topic in the NLP domain, the literature
lacks studies that systematically categorize and summarize backdoor attacks and defenses …

We introduce MARGE, a pre-trained sequence-to-sequence model learned with an
unsupervised multi-lingual multi-document paraphrasing objective. MARGE provides an …

Although neural models have achieved impressive results on several NLP benchmarks, little
is understood about the mechanisms they use to perform language tasks. Thus, much recent …

We investigate the extent to which individual attention heads in pretrained transformer
language models, such as BERT and RoBERTa, implicitly capture syntactic dependency …