A comprehensive review of data‐driven co‐speech gesture generation

S Nyatsanga, T Kucherenko, C Ahuja… - Computer Graphics …, 2023 - Wiley Online Library
Gestures that accompany speech are an essential part of natural and efficient embodied
human communication. The automatic generation of such co‐speech gestures is a long …

Digital body, identity and privacy in social virtual reality: A systematic review

J Lin, ME Latoschik - Frontiers in Virtual Reality, 2022 - frontiersin.org
Social Virtual Reality (social VR or SVR) provides digital spaces for diverse human activities,
social interactions, and embodied face-to-face encounters. While our digital bodies in SVR …

Listen, denoise, action! audio-driven motion synthesis with diffusion models

S Alexanderson, R Nagy, J Beskow… - ACM Transactions on …, 2023 - dl.acm.org
Diffusion models have experienced a surge of interest as highly expressive yet efficiently
trainable probabilistic models. We show that these models are an excellent fit for …

Frankmocap: A monocular 3d whole-body pose estimation system via regression and integration

Y Rong, T Shiratori, H Joo - Proceedings of the IEEE/CVF …, 2021 - openaccess.thecvf.com
Most existing monocular 3D pose estimation approaches only focus on a single body part,
neglecting the fact that the essential nuance of human motion is conveyed through a concert …

Learning hierarchical cross-modal association for co-speech gesture generation

X Liu, Q Wu, H Zhou, Y Xu, R Qian… - Proceedings of the …, 2022 - openaccess.thecvf.com
Generating speech-consistent body and gesture movements is a long-standing problem in
virtual avatar creation. Previous studies often synthesize pose movement in a holistic …

From audio to photoreal embodiment: Synthesizing humans in conversations

E Ng, J Romero, T Bagautdinov, S Bai… - Proceedings of the …, 2024 - openaccess.thecvf.com
We present a framework for generating full-bodied photorealistic avatars that gesture
according to the conversational dynamics of a dyadic interaction. Given speech audio we …

The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation

Y Yoon, P Wolfert, T Kucherenko, C Viegas… - Proceedings of the …, 2022 - dl.acm.org
This paper reports on the second GENEA Challenge to benchmark data-driven automatic co-
speech gesture generation. Participating teams used the same speech and motion dataset …

Manipnet: neural manipulation synthesis with a hand-object spatial representation

H Zhang, Y Ye, T Shiratori, T Komura - ACM Transactions on Graphics …, 2021 - dl.acm.org
Natural hand manipulations exhibit complex finger maneuvers adaptive to object shapes
and the tasks at hand. Learning dexterous manipulation from data in a brute force way …

The GENEA Challenge 2023: A large-scale evaluation of gesture generation models in monadic and dyadic settings

T Kucherenko, R Nagy, Y Yoon, J Woo… - Proceedings of the 25th …, 2023 - dl.acm.org
This paper reports on the GENEA Challenge 2023, in which participating teams built speech-
driven gesture-generation systems using the same speech and motion dataset, followed by …

Emotional speech-driven 3d body animation via disentangled latent diffusion

K Chhatre, N Athanasiou, G Becherini… - Proceedings of the …, 2024 - openaccess.thecvf.com
Existing methods for synthesizing 3D human gestures from speech have shown promising
results but they do not explicitly model the impact of emotions on the generated gestures …