Speech enhancement and speech separation are two related tasks, whose purpose is to
extract either one or more target speech signals, respectively, from a mixture of sounds …

Lip reading has gained popularity due to the proliferation of emerging real-world
applications. This article provides a comprehensive review of benchmark datasets available …

Prior works on improving speech quality with visual input typically study each type of
auditory distortion separately (eg, separation, inpainting, video-to-speech) and present …

Audio Packet Loss Concealment (PLC) is the hiding of gaps in audio streams caused by
data transmission failures in packet switched networks. This is a common problem, and of …

In this paper, we propose to make a systematic study on machines' multisensory perception
under attacks. We use the audio-visual event recognition task against multimodal …

We propose SpeechPainter, a model for filling in gaps of up to one second in speech
samples by leveraging an auxiliary textual input. We demonstrate that the model performs …

In this manuscript, we propose a novel method to perform audio inpainting, ie, the
restoration of audio signals presenting multiple missing parts. Audio inpainting can be …

Audio inpainting aims to reconstruct missing segments in corrupted recordings. Most of
existing methods produce plausible reconstructions when the gap lengths are short, but …

Audio-visual speech synthesis (AVSS) has garnered attention in recent years for its utility in
the realm of audio-visual learning. AVSS transforms one speaker's speech into another's …

Prior works on improving speech quality with visual input typically study each type of
auditory distortion separately (eg, separation, inpainting, video-to-speech) and present …