This paper explores a simple method for improving the zero-shot learning abilities of
language models. We show that instruction tuning--finetuning language models on a …

Pretraining large neural language models, such as BERT, has led to impressive gains on
many natural language processing (NLP) tasks. However, most pretraining efforts focus on …

The goal of text ranking is to generate an ordered list of texts retrieved from a corpus in
response to a query. Although the most common formulation of text ranking is search …

Deep neural networks excel at learning from labeled data and achieve state-of-the-art
resultson a wide array of Natural Language Processing tasks. In contrast, learning from …

Machine translation has made rapid advances in recent years. Millions of people are using it
today in online translation systems and mobile applications in order to communicate across …

The notion of" in-domain data" in NLP is often over-simplistic and vague, as textual data
varies in many nuanced linguistic aspects such as topic, style or level of formality. In …

We study the topmost weight matrix of neural network language models. We show that this
matrix constitutes a valid word embedding. When training language models, we recommend …

Neural machine translation is a recently proposed approach to machine translation. Unlike
the traditional statistical machine translation, the neural machine translation aims at building …

In this paper, we propose a novel neural network model called RNN Encoder-Decoder that
consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols …

Neural machine translation (NMT) is a deep learning based approach for machine
translation, which outperforms traditional statistical machine translation (SMT) and yields the …