Chatbots are intelligent conversational computer systems designed to mimic human
conversation to enable automated online guidance and support. The increased benefits of …

In recent years, conversational agents (CAs) have become ubiquitous and are a presence in
our daily routines. It seems that the technology has finally ripened to advance the use of CAs …

We introduce GPT-NeoX-20B, a 20 billion parameter autoregressive language model
trained on the Pile, whose weights will be made freely and openly available to the public …

We present the call for papers for the BabyLM Challenge: Sample-efficient pretraining on a
developmentally plausible corpus. This shared task is intended for participants with an …

As language models grow ever larger, the need for large-scale high-quality text datasets has
never been more pressing, especially in multilingual settings. The BigScience workshop, a 1 …

Abstract Natural Language Processing (NLP) is a branch of artificial intelligence that
involves the design and implementation of systems and algorithms able to interact through …

We present Branch-Train-Merge (BTM), a communication-efficient algorithm for
embarrassingly parallel training of large language models (LLMs). We show it is possible to …

This paper introduces mGPT, a multilingual variant of GPT-3, pretrained on 61 languages
from 25 linguistically diverse language families using Wikipedia and the C4 Corpus. We …

We present an easy and efficient method to extend existing sentence embedding models to
new languages. This allows to create multilingual versions from previously monolingual …

Rapid progress in machine learning for natural language processing has the potential to
transform debates about how humans learn language. However, the learning environments …