Large language models (LLMs) have come closest among all models to date to mastering
human language, yet opinions about their linguistic and cognitive capabilities remain split …

Transformer language models have received widespread public attention, yet their
generated text is often surprising even to NLP researchers. In this survey, we discuss over …

We present the call for papers for the BabyLM Challenge: Sample-efficient pretraining on a
developmentally plausible corpus. This shared task is intended for participants with an …

Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved
great success and become a milestone in the field of artificial intelligence (AI). Owing to …

Transformer-based models have pushed state of the art in many areas of NLP, but our
understanding of what is behind their success is still limited. This paper is the first survey of …

Rapid progress in machine learning for natural language processing has the potential to
transform debates about how humans learn language. However, the learning environments …

Language understanding research is held back by a failure to relate language to the
physical world it describes and to the social interactions it facilitates. Despite the incredible …

Natural language is characterized by compositionality: the meaning of a complex expression
is constructed from the meanings of its constituent parts. To facilitate the evaluation of the …

Modern deep neural networks achieve impressive performance in engineering applications
that require extensive linguistic skills, such as machine translation. This success has …

While state-of-the-art neural network models continue to achieve lower perplexity scores on
language modeling benchmarks, it remains unknown whether optimizing for broad …