Large motion model for unified multi-modal motion generation

M Zhang, D **, C Gu, F Hong, Z Cai, J Huang… - … on Computer Vision, 2024 - Springer
Human motion generation, a cornerstone technique in animation and video production, has
widespread applications in various tasks like text-to-motion and music-to-dance. Previous …

Emotional speech-driven 3d body animation via disentangled latent diffusion

K Chhatre, N Athanasiou, G Becherini… - Proceedings of the …, 2024 - openaccess.thecvf.com
Existing methods for synthesizing 3D human gestures from speech have shown promising
results but they do not explicitly model the impact of emotions on the generated gestures …

Chain of generation: Multi-modal gesture synthesis via cascaded conditional control

Z Xu, Y Zhang, S Yang, R Li, X Li - … of the AAAI Conference on Artificial …, 2024 - ojs.aaai.org
This study aims to improve the generation of 3D gestures by utilizing multimodal information
from human speech. Previous studies have focused on incorporating additional modalities …

The diffusestylegesture+ entry to the genea challenge 2023

S Yang, H Xue, Z Zhang, M Li, Z Wu, X Wu… - Proceedings of the 25th …, 2023 - dl.acm.org
In this paper, we introduce the DiffuseStyleGesture+, our solution for the Generation and
Evaluation of Non-verbal Behavior for Embodied Agents (GENEA) Challenge 2023, which …

Semantic Gesticulator: Semantics-Aware Co-Speech Gesture Synthesis

Z Zhang, T Ao, Y Zhang, Q Gao, C Lin… - ACM Transactions on …, 2024 - dl.acm.org
In this work, we present Semantic Gesticulator, a novel framework designed to synthesize
realistic gestures accompanying speech with strong semantic correspondence. Semantically …

BOTH2Hands: Inferring 3D Hands from Both Text Prompts and Body Dynamics

W Zhang, M Huang, Y Zhou, J Zhang… - Proceedings of the …, 2024 - openaccess.thecvf.com
The recently emerging text-to-motion advances have spired numerous attempts for
convenient and interactive human motion generation. Yet existing methods are largely …

Mambatalk: Efficient holistic gesture synthesis with selective state space models

Z Xu, Y Lin, H Han, S Yang, R Li, Y Zhang… - arxiv preprint arxiv …, 2024 - arxiv.org
Gesture synthesis is a vital realm of human-computer interaction, with wide-ranging
applications across various fields like film, robotics, and virtual reality. Recent advancements …

Towards Variable and Coordinated Holistic Co-Speech Motion Generation

Y Liu, Q Cao, Y Wen, H Jiang… - Proceedings of the IEEE …, 2024 - openaccess.thecvf.com
This paper addresses the problem of generating lifelike holistic co-speech motions for 3D
avatars focusing on two key aspects: variability and coordination. Variability allows the …

Diffugesture: Generating human gesture from two-person dialogue with diffusion models

W Zhao, L Hu, S Zhang - Companion Publication of the 25th International …, 2023 - dl.acm.org
This paper describes the DiffuGesture entry to the GENEA Challenge 2023. In this paper, we
utilize conditional diffusion models to formulate the gesture generation problem. The …

Combo: Co-speech holistic 3D human motion generation and efficient customizable adaptation in harmony

C Xu, M Sun, ZQ Cheng, F Wang, Y Liu, B Sun… - arxiv preprint arxiv …, 2024 - arxiv.org
In this paper, we propose a novel framework, Combo, for harmonious co-speech holistic 3D
human motion generation and efficient customizable adaption. In particular, we identify that …