Omg: Towards open-vocabulary motion generation via mixture of controllers

H Liang, J Bao, R Zhang, S Ren, Y Xu… - Proceedings of the …, 2024 - openaccess.thecvf.com
We have recently seen tremendous progress in realistic text-to-motion generation. Yet the
existing methods often fail or produce implausible motions with unseen text inputs which …

[HTML][HTML] Audio-Driven Facial Animation with Deep Learning: A Survey

D Jiang, J Chang, L You, S Bian, R Kosk, G Maguire - Information, 2024 - mdpi.com
Audio-driven facial animation is a rapidly evolving field that aims to generate realistic facial
expressions and lip movements synchronized with a given audio input. This survey provides …

Diffspeaker: Speech-driven 3d facial animation with diffusion transformer

Z Ma, X Zhu, G Qi, C Qian, Z Zhang, Z Lei - arxiv preprint arxiv …, 2024 - arxiv.org
Speech-driven 3D facial animation is important for many multimedia applications. Recent
work has shown promise in using either Diffusion models or Transformer architectures for …

Instructavatar: Text-guided emotion and motion control for avatar generation

Y Wang, J Guo, J Bai, R Yu, T He, X Tan, X Sun… - arxiv preprint arxiv …, 2024 - arxiv.org
Recent talking avatar generation models have made strides in achieving realistic and
accurate lip synchronization with the audio, but often fall short in controlling and conveying …

Learn2talk: 3d talking face learns from 2d talking face

Y Zhuang, B Cheng, Y Cheng, Y **… - … on Visualization and …, 2024 - ieeexplore.ieee.org
The speech-driven facial animation technology is generally categorized into two main types:
3D and 2D talking face. Both of these have garnered considerable research attention in …

Beyond talking–generating holistic 3d human dyadic motion for communication

M Sun, C Xu, X Jiang, Y Liu, B Sun… - International Journal of …, 2024 - Springer
In this paper, we introduce an innovative task focused on human communication, aiming to
generate 3D holistic human motions for both speakers and listeners. Central to our …

UniTalker: Scaling up Audio-Driven 3D Facial Animation through A Unified Model

X Fan, J Li, Z Lin, W **ao, L Yang - European Conference on Computer …, 2024 - Springer
Audio-driven 3D facial animation aims to map input audio to realistic facial motion. Despite
significant progress, limitations arise from inconsistent 3D annotations, restricting previous …

MMHead: Towards Fine-grained Multi-modal 3D Facial Animation

S Wu, Y Li, Y Yan, H Duan, Z Liu, G Zhai - Proceedings of the 32nd ACM …, 2024 - dl.acm.org
3D facial animation has attracted considerable attention due to its extensive applications in
the multimedia field. Audio-driven 3D facial animation has been widely explored with …

DICE: End-to-end Deformation Capture of Hand-Face Interactions from a Single Image

Q Wu, Z Dou, S Xu, S Shimada, C Wang, Z Yu… - arxiv preprint arxiv …, 2024 - arxiv.org
Reconstructing 3D hand-face interactions with deformations from a single image is a
challenging yet crucial task with broad applications in AR, VR, and gaming. The challenges …

ProbTalk3D: Non-Deterministic Emotion Controllable Speech-Driven 3D Facial Animation Synthesis Using VQ-VAE

S Wu, KI Haque, Z Yumak - Proceedings of the 17th ACM SIGGRAPH …, 2024 - dl.acm.org
Audio-driven 3D facial animation synthesis has been an active field of research with
attention from both academia and industry. While there are promising results in this area …