State of the art on monocular 3D face reconstruction, tracking, and applications

M Zollhöfer, J Thies, P Garrido, D Bradley… - Computer graphics …, 2018 - Wiley Online Library
The computer graphics and vision communities have dedicated long standing efforts in
building computerized tools for reconstructing, tracking, and analyzing human faces based …

Differentiable rendering: A survey

H Kato, D Beker, M Morariu, T Ando… - arxiv preprint arxiv …, 2020 - arxiv.org
Deep neural networks (DNNs) have shown remarkable performance improvements on
vision-related tasks such as object detection or image segmentation. Despite their success …

Dynamic neural radiance fields for monocular 4d facial avatar reconstruction

G Gafni, J Thies, M Zollhofer… - Proceedings of the …, 2021 - openaccess.thecvf.com
We present dynamic neural radiance fields for modeling the appearance and dynamics of a
human face. Digitally modeling and reconstructing a talking human is a key building-block …

Neural head avatars from monocular rgb videos

PW Grassal, M Prinzler, T Leistner… - Proceedings of the …, 2022 - openaccess.thecvf.com
Abstract We present Neural Head Avatars, a novel neural representation that explicitly
models the surface geometry and appearance of an animatable human avatar that can be …

Face2face: Real-time face capture and reenactment of rgb videos

J Thies, M Zollhofer, M Stamminger… - Proceedings of the …, 2016 - openaccess.thecvf.com
We present a novel approach for real-time facial reenactment of a monocular target video
sequence (eg, Youtube video). The source sequence is also a monocular video stream …

[PDF][PDF] Real-time expression transfer for facial reenactment.

J Thies, M Zollhöfer, M Nießner, L Valgaerts… - ACM Trans …, 2015 - niessnerlab.org
We present a method for the real-time transfer of facial expressions from an actor in a source
video to an actor in a target video, thus enabling the ad-hoc control of the facial expressions …

Real-time high-fidelity facial performance capture

C Cao, D Bradley, K Zhou, T Beeler - ACM Transactions on Graphics …, 2015 - dl.acm.org
We present the first real-time high-fidelity facial capture method. The core idea is to enhance
a global real-time face tracker, which provides a low-resolution face mesh, with local …

Facial performance sensing head-mounted display

H Li, L Trutoiu, K Olszewski, L Wei, T Trutna… - ACM Transactions on …, 2015 - dl.acm.org
There are currently no solutions for enabling direct face-to-face interaction between virtual
reality (VR) users wearing head-mounted displays (HMDs). The main challenge is that the …

High-fidelity 3d digital human head creation from rgb-d selfies

L Bao, X Lin, Y Chen, H Zhang, S Wang, X Zhe… - ACM Transactions on …, 2021 - dl.acm.org
We present a fully automatic system that can produce high-fidelity, photo-realistic three-
dimensional (3D) digital human heads with a consumer RGB-D selfie camera. The system …

High-fidelity facial and speech animation for VR HMDs

K Olszewski, JJ Lim, S Saito, H Li - ACM Transactions on Graphics (TOG), 2016 - dl.acm.org
Significant challenges currently prohibit expressive interaction in virtual reality (VR).
Occlusions introduced by head-mounted displays (HMDs) make existing facial tracking …