[HTML][HTML] Deepfake attacks: Generation, detection, datasets, challenges, and research directions

A Naitali, M Ridouani, F Salahdine, N Kaabouch - Computers, 2023‏ - mdpi.com
Recent years have seen a substantial increase in interest in deepfakes, a fast-develo**
field at the nexus of artificial intelligence and multimedia. These artificial media creations …

Deep learning for visual speech analysis: A survey

C Sheng, G Kuang, L Bai, C Hou, Y Guo… - … on Pattern Analysis …, 2024‏ - ieeexplore.ieee.org
Visual speech, referring to the visual domain of speech, has attracted increasing attention
due to its wide applications, such as public security, medical treatment, military defense, and …

Codetalker: Speech-driven 3d facial animation with discrete motion prior

J **ng, M **a, Y Zhang, X Cun… - Proceedings of the …, 2023‏ - openaccess.thecvf.com
Speech-driven 3D facial animation has been widely studied, yet there is still a gap to
achieving realism and vividness due to the highly ill-posed nature and scarcity of audio …

[PDF][PDF] Multimodal image synthesis and editing: A survey

F Zhan, Y Yu, R Wu, J Zhang, S Lu, L Liu… - arxiv preprint arxiv …, 2022‏ - pure.mpg.de
As information exists in various modalities in real world, effective interaction and fusion
among multimodal information plays a key role for the creation and perception of multimodal …

Pose-controllable talking face generation by implicitly modularized audio-visual representation

H Zhou, Y Sun, W Wu, CC Loy… - Proceedings of the …, 2021‏ - openaccess.thecvf.com
While accurate lip synchronization has been achieved for arbitrary-subject audio-driven
talking face generation, the problem of how to efficiently drive the head pose remains …

Ad-nerf: Audio driven neural radiance fields for talking head synthesis

Y Guo, K Chen, S Liang, YJ Liu… - Proceedings of the …, 2021‏ - openaccess.thecvf.com
Generating high-fidelity talking head video by fitting with the input audio sequence is a
challenging problem that receives considerable attentions recently. In this paper, we …

Stylesync: High-fidelity generalized and personalized lip sync in style-based generator

J Guan, Z Zhang, H Zhou, T Hu… - Proceedings of the …, 2023‏ - openaccess.thecvf.com
Despite recent advances in syncing lip movements with any audio waves, current methods
still struggle to balance generation quality and the model's generalization ability. Previous …

Pirenderer: Controllable portrait image generation via semantic neural rendering

Y Ren, G Li, Y Chen, TH Li… - Proceedings of the IEEE …, 2021‏ - openaccess.thecvf.com
Generating portrait images by controlling the motions of existing faces is an important task of
great consequence to social media industries. For easy use and intuitive control …

Expressive talking head generation with granular audio-visual control

B Liang, Y Pan, Z Guo, H Zhou… - Proceedings of the …, 2022‏ - openaccess.thecvf.com
Generating expressive talking heads is essential for creating virtual humans. However,
existing one-or few-shot methods focus on lip-sync and head motion, ignoring the emotional …

Styleheat: One-shot high-resolution editable talking face generation via pre-trained stylegan

F Yin, Y Zhang, X Cun, M Cao, Y Fan, X Wang… - European conference on …, 2022‏ - Springer
One-shot talking face generation aims at synthesizing a high-quality talking face video from
an arbitrary portrait image, driven by a video or an audio segment. In this work, we provide a …