Forecasting Credit Ratings: A Case Study where Traditional Methods Outperform Generative LLMs
Abstract Large Language Models (LLMs) have been shown to perform well for many
downstream tasks. Transfer learning can enable LLMs to acquire skills that were not …
downstream tasks. Transfer learning can enable LLMs to acquire skills that were not …
Measuring memorization through probabilistic discoverable extraction
Large language models (LLMs) are susceptible to memorizing training data, raising
concerns due to the potential extraction of sensitive information. Current methods to …
concerns due to the potential extraction of sensitive information. Current methods to …
Paying Attention to Facts: Quantifying the Knowledge Capacity of Attention Layers
LZ Wong - arxiv preprint arxiv:2502.05076, 2025 - arxiv.org
In this paper, we investigate the ability of single-layer attention-only transformers (ie
attention layers) to memorize facts contained in databases from a linear-algebraic …
attention layers) to memorize facts contained in databases from a linear-algebraic …
Sequence-Level Analysis of Leakage Risk of Training Data in Large Language Models
T Tiwari, GE Suh - arxiv preprint arxiv:2412.11302, 2024 - arxiv.org
This work advocates for the use of sequence level probabilities for quantifying the risk of
extraction training data from Large Language Models (LLMs) as they provide much finer …
extraction training data from Large Language Models (LLMs) as they provide much finer …