It has long been argued that only humans could produce and understand language. But
now, for the first time, artificial language models (LMs) achieve this feat. Here we survey the …

In natural language processing, short-text semantic similarity (STSS) is a very prominent
field. It has a significant impact on a broad range of applications, such as question …

Transformer-based models have pushed state of the art in many areas of NLP, but our
understanding of what is behind their success is still limited. This paper is the first survey of …

Transformer architectures have proven to learn useful representations for protein
classification and generation tasks. However, these representations present challenges in …

Does the effectiveness of neural language models derive entirely from accurate modeling of
surface word co-occurrence statistics, or do these models represent and reason about the …

Abstract Machines have achieved a broad and growing set of linguistic competencies,
thanks to recent progress in Natural Language Processing (NLP). Psychologists have …

Vector-based word representation paradigms situate lexical meaning at different levels of
abstraction. Distributional and static embedding models generate a single vector per word …

Distributional semantics has deeply changed in the last decades. First, predict models stole
the thunder from traditional count ones, and more recently both of them were replaced in …

Pre-trained language models (LMs) encode rich information about linguistic structure but
their knowledge about lexical polysemy remains unclear. We propose a novel experimental …

Distributional semantics develops theories and methods to represent the meaning of natural
language expressions, with vectors encoding their statistical distribution in linguistic …