We propose emotion2vec, a universal speech emotion representation model. emotion2vec
is pre-trained on open-source unlabeled emotion data through self-supervised online …

In recent years, foundation models (FMs) such as large language models (LLMs) and latent
diffusion models (LDMs) have profoundly impacted diverse sectors, including music. This …

The rapid development of single-modal pre-training has prompted researchers to pay more
attention to cross-modal pre-training methods. In this paper, we propose a unified-modal …

How to boost speech pre-training with textual data is an unsolved problem due to the fact
that speech and text are very different modalities with distinct characteristics. In this paper …

Self-supervised learning (SSL) has led to great strides in speech processing. However, the
resources needed to train these models has become prohibitively large as they continue to …

In this paper, we provide a new perspective on self-supervised speech models from how the
self-training targets are obtained. We generalize the targets extractor into Offline Targets …

The excellent generalization ability of self-supervised learning (SSL) for speech foundation
models has garnered significant attention. HuBERT is a successful example that utilizes …

Recent years have witnessed significant advancements in self-supervised learning (SSL)
methods for speech-processing tasks. Various speech-based SSL models have been …

In this work, we present a simple but effective method, CTCBERT, for advancing hidden-unit
BERT (HuBERT). HuBERT applies a frame-level cross-entropy (CE) loss, which is similar to …

Self-supervised pre-training has been successful in both text and speech processing.
Speech and text offer different but complementary information. The question is whether we …