Current self-supervised learning algorithms are often modality-specific and require large
amounts of computational resources. To address these issues, we increase the training …

In recent years, neural models learned through self-supervised pretraining on large scale
multilingual text or speech data have exhibited promising results for underresourced …

Compressing self-supervised models has become increasingly necessary, as self-
supervised models become larger. While previous approaches have primarily focused on …

Speech representations learned with self-supervised learning (SSL) have the potential to
significantly improve the performance of a number of audio applications, especially when …

Self-supervised learning enables the training of large neural models without the need for
large, labeled datasets. It has been generating breakthroughs in several fields, including …

Self-supervised learning has achieved impressive results in speech processing, but current
models are computationally expensive, generating environmental concerns because of their …

Despite their impressive success, training foundation models remains computationally
costly. This paper investigates how to efficiently train speech foundation models with self …

Sustainable artificial intelligence focuses on data, hardware, and algorithms to make
machine learning models more environmentally responsible. In particular, machine learning …

The sequence length along the time axis is often the dominant factor of the computation in
speech processing. Works have been proposed to reduce the sequence length for lowering …

In this paper, we show that a simple audio language model can achieve comparable
inference efficiency to more complicated pre-trained models with speech transformer …