The recent release of ChatGPT, a chat bot research project/product of natural language
processing (NLP) by OpenAI, stirs up a sensation among both the general public and …

Deep learning--based models have surpassed classical machine learning--based
approaches in various text classification tasks, including sentiment analysis, news …

Pretraining Large Language Models (LLMs) on large corpora of textual data is now a
standard paradigm. When using these LLMs for many downstream applications, it is …

Selecting a suitable pretraining dataset is crucial for both general-domain (eg, GPT-3) and
domain-specific (eg, Codex) language models (LMs). We formalize this problem as selecting …

Large language models (LLMs) can be used to generate text data for training and evaluating
other models. However, creating high-quality datasets with LLMs can be challenging. In this …

We explore how continued pre-training on domain-specific corpora influences large
language models, revealing that training on the raw corpora endows the model with domain …

Language models pretrained on text from a wide variety of sources form the foundation of
today's NLP. In light of the success of these broad-coverage models, we investigate whether …

Common acquisition functions for active learning use either uncertainty or diversity
sampling, aiming to select difficult and diverse data points from the pool of unlabeled data …

Recent neural language models have taken a significant step forward in producing
remarkably controllable, fluent, and grammatical text. Although studies have found that AI …

Active learning strives to reduce annotation costs by choosing the most critical examples to
label. Typically, the active learning strategy is contingent on the classification model. For …