This article provides a comprehensive survey of contemporary language modeling
approaches within the realm of natural language processing (NLP) tasks. This paper …

Knowledge distillation (KD) has been widely adopted to compress large language models
(LLMs). Existing KD methods investigate various divergence measures including the …

Abstract Language models (LMs) are statistical models trained to assign probability to
humangenerated text. As such, it is reasonable to question whether they approximate …

Neural language models are probabilistic models of human text. They are predominantly
trained using maximum likelihood estimation (MLE), which is equivalent to minimizing the …

We present a setup for training, evaluating and interpreting neural language models, that
uses artificial, language-like data. The data is generated using a massive probabilistic …

Maximum likelihood estimation (MLE) is a statistical method used to estimate the parameters
of a probability distribution that best explain the observed data. In the context of text …

Zero-shot relation triplet extraction (ZeroRTE) aims to extract relation triplets from
unstructured text under zero-shot conditions, where the relation sets in the training and …

Linguistic steganography embeds secret information in seemingly innocent texts,
safeguarding privacy in surveillance environments. Generative linguistic steganography …

Language models (LMs) are statistical models trained to assign probability to human-
generated text. As such, it is reasonable to question whether they approximate linguistic …

When we speak, write or listen, we continuously make predictions based on our knowledge
of a language's grammar. Remarkably, children acquire this grammatical knowledge within …