We introduce a self-supervised speech pre-training method called TERA, which stands for
Transformer Encoder Representations from Alteration. Recent approaches often learn by …

Progress in speech processing has been facilitated by shared datasets and benchmarks.
Historically these have focused on automatic speech recognition (ASR), speaker …

Fine-tuning pre-trained models for downstream tasks is mainstream in deep learning.
However, the pre-trained models are limited to be fine-tuned by data from a specific …

The widespread application and rapid development of ChatGPT are disrupting traditional
models across industries, bringing revolutionary changes to electronic records …

Spoken language understanding (SLU) refers to the process of inferring the semantic
information from audio signals. While the neural transformers consistently deliver the best …

Much recent work on Spoken Language Understanding (SLU) is limited in at least one of
three ways: models were trained on oracle text input and neglected ASR errors, models …

End-to-end (E2E) spoken language understanding (SLU) can infer semantics directly from
speech signal without cascading an automatic speech recognizer (ASR) with a natural …

Language model pre-training has shown promising results in various downstream tasks. In
this context, we introduce a cross-modal pre-trained language model, called Speech-Text …

Self-supervised Audio Transformers (SAT) enable great success in many downstream
speech applications like ASR, but how they work has not been widely explored yet. In this …

Abstract Voice Assistants such as Alexa, Siri, and Google Assistant typically use a two-stage
Spoken Language Understanding pipeline; first, an Automatic Speech Recognition (ASR) …