We propose a cross-lingual neural codec language model, VALL-E X, for cross-lingual
speech synthesis. Specifically, we extend VALL-E and train a multi-lingual conditional codec …

Pre-training speech models on large volumes of data has achieved remarkable success.
OpenAI Whisper is a multilingual multitask model trained on 680k hours of supervised …

Abstract Speech-to-Text (ST) translation pertains to the task of converting speech signals in
one language to text in another language. It finds its application in various domains, such as …

How to solve the data scarcity problem for end-to-end speech-to-text translation (ST)? It's
well known that data augmentation is an efficient method to improve performance for many …

This paper reports on the shared tasks organized by the 20th IWSLT Conference. The
shared tasks address 9 scientific challenges in spoken language translation: simultaneous …

Given the great success of large language models (LLMs) across various tasks, in this
paper, we introduce LLM-ST, a novel and effective speech translation model constructed …

Language models (LMs) have recently flourished in natural language processing and
computer vision, generating high-fidelity texts or images in various tasks. In contrast, the …

The Open Whisper-style Speech Model (OWSM) series was introduced to achieve full
transparency in building advanced speech-to-text (S2T) foundation models. To this end …

Automatic speech recognition (ASR) and speech translation (ST) can both use neural
transducers as the model structure. It is thus possible to use a single transducer model to …

With the rise of Speech Large Language Models (Speech LLMs), there has been growing
interest in discrete speech tokens for their ability to integrate with text-based tokens …