Abstract Pretrained Foundation Models (PFMs) are regarded as the foundation for various
downstream tasks across different data modalities. A PFM (eg, BERT, ChatGPT, GPT-4) is …

In machine learning, two approaches outperform traditional algorithms: ensemble learning
and deep learning. The former refers to methods that integrate multiple base models in the …

Language models (LMs) are becoming the foundation for almost all major language
technologies, but their capabilities, limitations, and risks are not well understood. We present …

This paper presents SimCSE, a simple contrastive learning framework that greatly advances
state-of-the-art sentence embeddings. We first describe an unsupervised approach, which …

BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019) has set a new state-of-the-art
performance on sentence-pair regression tasks like semantic textual similarity (STS) …

When primed with only a handful of training samples, very large, pretrained language
models such as GPT-3 have shown competitive results when compared to fully-supervised …

While supervised learning has enabled great progress in many applications, unsupervised
learning has not seen such widespread adoption, and remains an important and …

Deep learning--based models have surpassed classical machine learning--based
approaches in various text classification tasks, including sentiment analysis, news …

While BERT is an effective method for learning monolingual sentence embeddings for
semantic similarity and embedding based transfer learning (Reimers and Gurevych, 2019) …

A prominent technique for self-supervised representation learning has been to contrast
semantically similar and dissimilar pairs of samples. Without access to labels, dissimilar …