Despite pre-training's progress in many important NLP tasks, it remains to explore effective
pre-training strategies for dense retrieval. In this paper, we propose RetroMAE, a new …

Abstract Recently, Language Models (LMs) instruction-tuned on multiple tasks, also known
as multitask-prompted fine-tuning (MT), have shown capabilities to generalize to unseen …

High-quality text embedding is pivotal in improving semantic textual similarity (STS) tasks,
which are crucial components in Large Language Model (LLM) applications. However, a …

In this paper, we propose SimLM (Similarity matching with Language Model pre-training), a
simple yet effective pre-training method for dense passage retrieval. It employs a simple …

Contrastive learning has become a popular approach in natural language processing,
particularly for the learning of sentence embeddings. However, the discrete nature of natural …

Text-based person search aims to retrieve the specified person images given a textual
description. The key to tackling such a challenging task is to learn powerful multi-modal …

Large language models (LLMs) have recently garnered significant interest. With in-context
learning, LLMs achieve impressive results in various natural language tasks. However, the …

Contrastive learning has been extensively studied in sentence embedding learning, which
assumes that the embeddings of different views of the same sentence are closer. The …

Sentence embedding, which aims to learn an effective representation of the sentence, is
beneficial for downstream tasks. By utilizing contrastive learning, most recent sentence …

Contrastive learning (CL) benefits the training of sequential recommendation models with
informative self-supervision signals. Existing solutions apply general sequential data …