Does negative sampling matter? a review with insights into its theory and applications

Z Yang, M Ding, T Huang, Y Cen, J Song… - … on Pattern Analysis …, 2024 - ieeexplore.ieee.org
Negative sampling has swiftly risen to prominence as a focal point of research, with wide-
ranging applications spanning machine learning, computer vision, natural language …

Contrastive learning models for sentence representations

L Xu, H **e, Z Li, FL Wang, W Wang, Q Li - ACM Transactions on …, 2023 - dl.acm.org
Sentence representation learning is a crucial task in natural language processing, as the
quality of learned representations directly influences downstream tasks, such as sentence …

Heterogeneous contrastive learning for foundation models and beyond

L Zheng, B **g, Z Li, H Tong, J He - Proceedings of the 30th ACM …, 2024 - dl.acm.org
In the era of big data and Artificial Intelligence, an emerging paradigm is to utilize contrastive
self-supervised learning to model large-scale heterogeneous data. Many existing foundation …

Whitenedcse: Whitening-based contrastive learning of sentence embeddings

W Zhuo, Y Sun, X Wang, L Zhu… - Proceedings of the 61st …, 2023 - aclanthology.org
This paper presents a whitening-based contrastive learning method for sentence
embedding learning (WhitenedCSE), which combines contrastive learning with a novel …

[HTML][HTML] Contrastive sentence representation learning with adaptive false negative cancellation

L Xu, H **e, FL Wang, X Tao, W Wang, Q Li - Information Fusion, 2024 - Elsevier
Contrastive sentence representation learning has made great progress thanks to a range of
text augmentation strategies and hard negative sampling techniques. However, most studies …

Contrastive pre-training with adversarial perturbations for check-in sequence representation learning

L Gong, Y Lin, S Guo, Y Lin, T Wang, E Zheng… - Proceedings of the …, 2023 - ojs.aaai.org
A core step of mining human mobility data is to learn accurate representations for user-
generated check-in sequences. The learned representations should be able to fully describe …

micse: Mutual information contrastive learning for low-shot sentence embeddings

T Klein, M Nabi - arxiv preprint arxiv:2211.04928, 2022 - arxiv.org
This paper presents miCSE, a mutual information-based contrastive learning framework that
significantly advances the state-of-the-art in few-shot sentence embedding. The proposed …

Detective: Detecting ai-generated text via multi-level contrastive learning

X Guo, S Zhang, Y He, T Zhang, W Feng… - arxiv preprint arxiv …, 2024 - arxiv.org
Current techniques for detecting AI-generated text are largely confined to manual feature
crafting and supervised binary classification paradigms. These methodologies typically lead …

Contraclm: Contrastive learning for causal language model

N Jain, D Zhang, WU Ahmad, Z Wang, F Nan… - arxiv preprint arxiv …, 2022 - arxiv.org
Despite exciting progress in causal language models, the expressiveness of the
representations is largely limited due to poor discrimination ability. To remedy this issue, we …

EMMA-X: an EM-like multilingual pre-training algorithm for cross-lingual representation learning

P Guo, X Wei, Y Hu, B Yang, D Liu… - Advances in Neural …, 2023 - proceedings.neurips.cc
Expressing universal semantics common to all languages is helpful to understand the
meanings of complex and culture-specific sentences. The research theme underlying this …