Distributions in Semantic Space

K Selby - 2024 - uwspace.uwaterloo.ca
This thesis is an investigation of the powerful and flexible applications of analyzing empirical
distributions of vectors within latent spaces. These methods have historically been applied …

Graphmax for Text Generation

B Liu, G Yin - Journal of Artificial Intelligence Research, 2023 - jair.org
In text generation, a large language model (LM) makes a choice of each new word based
only on the former selection of its context using the softmax function. Nevertheless, the link …

[PDF][PDF] Modeling the Multi-mode Distribution in Self-Supervised Language Models

HS Chang - 2022 - core.ac.uk
Recently, researchers have found that transformer-based language models (LMs), such as
GPT-2, can predict the next word distribution better as their sizes grow [177, 21, 97] …

A graph total variation regularized softmax for text generation

L Bin, W Liang, G Yin - CoRR, 2021 - openreview.net
In text generation, a large language model (LM) makes a choice of each new word based
only on the former selection of its context using the softmax function. Nevertheless, the link …

Robust Embeddings Via Distributions

KA Selby, Y Wang, R Wang, P Passban… - arxiv preprint arxiv …, 2021 - arxiv.org
Despite recent monumental advances in the field, many Natural Language Processing
(NLP) models still struggle to perform adequately on noisy domains. We propose a novel …