State of the art on monocular 3D face reconstruction, tracking, and applications

M Zollhöfer, J Thies, P Garrido, D Bradley… - Computer graphics …, 2018 - Wiley Online Library
The computer graphics and vision communities have dedicated long standing efforts in
building computerized tools for reconstructing, tracking, and analyzing human faces based …

Emotalk: Speech-driven emotional disentanglement for 3d face animation

Z Peng, H Wu, Z Song, H Xu, X Zhu… - Proceedings of the …, 2023 - openaccess.thecvf.com
Speech-driven 3D face animation aims to generate realistic facial expressions that match
the speech content and emotion. However, existing methods often neglect emotional facial …

Audio-driven facial animation by joint end-to-end learning of pose and emotion

T Karras, T Aila, S Laine, A Herva… - ACM Transactions on …, 2017 - dl.acm.org
We present a machine learning technique for driving 3D facial animation by audio input in
real time and with low latency. Our deep neural network learns a map** from input …

Deep appearance models for face rendering

S Lombardi, J Saragih, T Simon, Y Sheikh - ACM Transactions on …, 2018 - dl.acm.org
We introduce a deep appearance model for rendering the human face. Inspired by Active
Appearance Models, we develop a data-driven rendering pipeline that learns a joint …

Lipsync3d: Data-efficient learning of personalized 3d talking faces from video using pose and lighting normalization

A Lahiri, V Kwatra, C Frueh, J Lewis… - Proceedings of the …, 2021 - openaccess.thecvf.com
In this paper, we present a video-based learning framework for animating personalized 3D
talking faces from audio. We introduce two training-time data normalizations that significantly …

Do we really need to collect millions of faces for effective face recognition?

I Masi, AT Trần, T Hassner, JT Leksut… - Computer Vision–ECCV …, 2016 - Springer
Face recognition capabilities have recently made extraordinary leaps. Though this progress
is at least partially due to ballooning training set sizes–huge numbers of face images …

3d morphable face models—past, present, and future

B Egger, WAP Smith, A Tewari, S Wuhrer… - ACM Transactions on …, 2020 - dl.acm.org
In this article, we provide a detailed survey of 3D Morphable Face Models over the 20 years
since they were first proposed. The challenges in building and applying these models …

Pixel codec avatars

S Ma, T Simon, J Saragih, D Wang… - Proceedings of the …, 2021 - openaccess.thecvf.com
Telecommunication with photorealistic avatars in virtual or augmented reality is a promising
path for achieving authentic face-to-face communication in 3D over remote physical …

Reconstruction of personalized 3D face rigs from monocular video

P Garrido, M Zollhöfer, D Casas, L Valgaerts… - ACM Transactions on …, 2016 - dl.acm.org
We present a novel approach for the automatic creation of a personalized high-quality 3D
face rig of an actor from just monocular video data (eg, vintage movies). Our rig is based on …

Dynamic 3D avatar creation from hand-held video input

AE Ichim, S Bouaziz, M Pauly - ACM Transactions on Graphics (ToG), 2015 - dl.acm.org
We present a complete pipeline for creating fully rigged, personalized 3D facial avatars from
hand-held video. Our system faithfully recovers facial expression dynamics of the user by …