For decades, cognitive scientists have debated what kind of representation might
characterize human concepts. Whatever the format of the representation, it must allow for the …

Extensive evaluation on a large number of word embedding models for language
processing applications is conducted in this work. First, we introduce popular word …

One of the most remarkable properties of word embeddings is the fact that they capture
certain types of semantic and syntactic relationships. Recently, pre-trained language models …

Analogies such as man is to king as woman is to X are often used to illustrate the amazing
power of word embeddings. Concurrently, they have also been used to expose how strongly …

Distributional semantics has deeply changed in the last decades. First, predict models stole
the thunder from traditional count ones, and more recently both of them were replaced in …

Artificial intelligence and machine learning (AI/ML) research has aimed to achieve human-
level performance in tasks that require understanding and decision making. Although major …

As a part of natural language processing (NLP), the intent of topic modeling is to identify
topics in textual corpora with limited human input. Current topic modeling techniques, like …

Over the last decade, AI models of language and word meaning have been dominated by
what we might call a statistics-of-occurrence, strategy: these models are deep neural net …

Vector-based word representation paradigms situate lexical meaning at different levels of
abstraction. Distributional and static embedding models generate a single vector per word …

This paper examines the encoding of analogy in large-scale pretrained language models,
such as BERT and GPT-2. Existing analogy datasets typically focus on a limited set of …