The pre-dominant approach to language modeling to date is based on recurrent neural
networks. Their success on this task is often linked to their ability to capture unbounded …

Neural networks are a family of powerful machine learning models and this book focuses on
their application to natural language data. The first half of the book (Parts I and II) covers the …

We consider the problem of producing compact architectures for text classification, such that
the full model fits in a limited amount of memory. After considering different solutions …

Deep Learning has enabled many advances in machine learning applications in the last few
years. However, since current Deep Learning algorithms require much energy for …

We consider the large-scale query-document retrieval problem: given a query (eg, a
question), return the set of relevant documents (eg, paragraphs containing the answer) from …

Jurassic-1 is a pair of auto-regressive language models recently released by AI21 Labs,
consisting of J1-Jumbo, a 178B-parameter model, and J1-Large, a 7B-parameter model. We …

Neural ranking models for information retrieval (IR) use shallow or deep neural networks to
rank search results in response to a query. Traditional learning to rank models employ …

Existing language models (LMs) predict tokens with a softmax over a finite vocabulary,
which can make it difficult to predict rare tokens or phrases. We introduce NPM, the first …

Convolutional networks trained on large supervised datasets produce visual features which
form the basis for the state-of-the-art in many computer-vision problems. Further …

Recurrent Neural Networks (RNN) are widely used to solve a variety of problems and as the
quantity of data and the amount of available compute have increased, so have model sizes …