A Systematic Literature Review: Facial Expression and Lip Movement Synchronization of an Audio Track

MH Alshahrani, MS Maashi - IEEE Access, 2024 - ieeexplore.ieee.org
This systematic literature review (SLR) explores the topic of Facial Expression and Lip
Movement Synchronization of an Audio Track in the context of Automatic Dubbing. This SLR …

Dubbing in practice: A large scale study of human localization with insights for automatic dubbing

W Brannon, Y Virkar, B Thompson - Transactions of the Association …, 2023 - direct.mit.edu
We investigate how humans perform the task of dubbing video content from one language
into another, leveraging a novel corpus of 319.57 hours of video from 54 professionally …

Videodubber: Machine translation with speech-aware length control for video dubbing

Y Wu, J Guo, X Tan, C Zhang, B Li, R Song… - Proceedings of the …, 2023 - ojs.aaai.org
Video dubbing aims to translate the original speech in a film or television program into the
speech in a target language, which can be achieved with a cascaded system consisting of …

From speech-to-speech translation to automatic dubbing

M Federico, R Enyedi, R Barra-Chicote, R Giri… - arxiv preprint arxiv …, 2020 - arxiv.org
We present enhancements to a speech-to-speech translation pipeline in order to perform
automatic dubbing. Our architecture features neural machine translation generating output …

Isometric mt: Neural machine translation for automatic dubbing

SM Lakew, Y Virkar, P Mathur… - ICASSP 2022-2022 …, 2022 - ieeexplore.ieee.org
Automatic dubbing (AD) is among the machine translation (MT) use cases where
translations should match a given length to allow for synchronicity between source and …

Duration modeling of neural tts for automatic dubbing

J Effendi, Y Virkar, R Barra-Chicote… - ICASSP 2022-2022 …, 2022 - ieeexplore.ieee.org
Automatic dubbing (AD) addresses the problem of translating speech in a video with speech
in another language while preserving the viewer experience. A most important requirement …

[HTML][HTML] Evaluating and optimizing prosodic alignment for automatic dubbing

M Federico, Y Virkar, R Enyedi, R Barra-Chicote - 2020 - amazon.science
Automatic dubbing aims at replacing all speech contained in a video with speech in a
different language, so that the result sounds and looks as natural as the original. Hence, in …

Direct speech translation for automatic subtitling

S Papi, M Gaido, A Karakanta, M Cettolo… - Transactions of the …, 2023 - direct.mit.edu
Automatic subtitling is the task of automatically translating the speech of audiovisual content
into short pieces of timed text, ie, subtitles and their corresponding timestamps. The …

Key challenges in using automatic dubbing to translate educational YouTube videos

R Baños - Linguistica Antverpiensia, New Series–Themes in …, 2023 - lans-tts.uantwerpen.be
The dubbing industry has been particularly concerned with how to cope with the higher
demand in dubbing experienced over the past few years. Media companies have invested in …

Machine translation verbosity control for automatic dubbing

SM Lakew, M Federico, Y Wang… - ICASSP 2021-2021 …, 2021 - ieeexplore.ieee.org
Automatic dubbing aims at seamlessly replacing the speech in a video document with
synthetic speech in a different language. The task implies many challenges, one of which is …