Word embeddings quantify 100 years of gender and ethnic stereotypes
Word embeddings are a powerful machine-learning framework that represents each English
word by a vector. The geometric relationship between these vectors captures meaningful …
word by a vector. The geometric relationship between these vectors captures meaningful …
Dynamic word embeddings
We present a probabilistic language model for time-stamped text data which tracks the
semantic evolution of individual words over time. The model represents words and contexts …
semantic evolution of individual words over time. The model represents words and contexts …
On the origins of linear representations in large language models
Recent works have argued that high-level semantic concepts are encoded" linearly" in the
representation space of large language models. In this work, we study the origins of such …
representation space of large language models. In this work, we study the origins of such …
Interpretable word embeddings via informative priors
Word embeddings have demonstrated strong performance on NLP tasks. However, lack of
interpretability and the unsupervised nature of word embeddings have limited their use …
interpretability and the unsupervised nature of word embeddings have limited their use …
Case vectors: Spatial representations of the law using document embeddings
Recent work in natural language processing represents language objects (words and
documents) as dense vectors that encode the relations between those objects. This paper …
documents) as dense vectors that encode the relations between those objects. This paper …
Learning dynamic author representations with temporal language models
E Delasalles, S Lamprier… - 2019 IEEE International …, 2019 - ieeexplore.ieee.org
Language models are at the heart of numerous works, notably in the text mining and
information retrieval communities. These statistical models aim at extracting word …
information retrieval communities. These statistical models aim at extracting word …
Dynamic word embeddings
Abstract Systems, methods, and articles of manufacture to perform an operation comprising
deriving, based on a corpus of electronic text, a machine learning data model that …
deriving, based on a corpus of electronic text, a machine learning data model that …
Improving optimization for models with continuous symmetry breaking
Many loss functions in representation learning are invariant under a continuous symmetry
transformation. For example, the loss function of word embeddings (Mikolov et al., 2013) …
transformation. For example, the loss function of word embeddings (Mikolov et al., 2013) …
A dynamic embedding model of the media landscape
Information about world events is disseminated through a wide variety of news channels,
each with specific considerations in the choice of their reporting. Although the multiplicity of …
each with specific considerations in the choice of their reporting. Although the multiplicity of …
The Geometry of Categorical and Hierarchical Concepts in Large Language Models
Understanding how semantic meaning is encoded in the representation spaces of large
language models is a fundamental problem in interpretability. In this paper, we study the two …
language models is a fundamental problem in interpretability. In this paper, we study the two …