Semantic Gesticulator: Semantics-Aware Co-Speech Gesture Synthesis

Z Zhang, T Ao, Y Zhang, Q Gao, C Lin… - ACM Transactions on …, 2024 - dl.acm.org
In this work, we present Semantic Gesticulator, a novel framework designed to synthesize
realistic gestures accompanying speech with strong semantic correspondence. Semantically …

Mambatalk: Efficient holistic gesture synthesis with selective state space models

Z Xu, Y Lin, H Han, S Yang, R Li, Y Zhang… - arxiv preprint arxiv …, 2024 - arxiv.org
Gesture synthesis is a vital realm of human-computer interaction, with wide-ranging
applications across various fields like film, robotics, and virtual reality. Recent advancements …

DisMouse: Disentangling Information from Mouse Movement Data

G Zhang, Z Hu, A Bulling - Proceedings of the 37th Annual ACM …, 2024 - dl.acm.org
Mouse movement data contain rich information about users, performed tasks, and user
interfaces, but separating the respective components remains challenging and unexplored …

EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling

H Liu, Z Zhu, G Becherini, Y Peng… - Proceedings of the …, 2024 - openaccess.thecvf.com
We propose EMAGE a framework to generate full-body human gestures from audio and
masked gestures encompassing facial local body hands and global movements. To achieve …

Enabling synergistic full-body control in prompt-based co-speech motion generation

B Chen, Y Li, YX Ding, T Shao, K Zhou - Proceedings of the 32nd ACM …, 2024 - dl.acm.org
Current co-speech motion generation approaches usually focus on upper body gestures
following speech contents only, while lacking supporting the elaborate control of synergistic …

Mdt-a2g: Exploring masked diffusion transformers for co-speech gesture generation

X Mao, Z Jiang, Q Wang, C Fu, J Zhang, J Wu… - Proceedings of the …, 2024 - dl.acm.org
Recent advancements in the field of Diffusion Transformers have substantially improved the
generation of high-quality 2D images, 3D videos, and 3D shapes. However, the …

Force: Dataset and method for intuitive physics guided human-object interaction

X Zhang, B Lal Bhatnagar, S Starke, I Petrov… - arxiv e …, 2024 - ui.adsabs.harvard.edu
Interactions between human and objects are influenced not only by the object's pose and
shape, but also by physical attributes such as object mass and surface friction. They …

Emage: Towards unified holistic co-speech gesture generation via masked audio gesture modeling

H Liu, Z Zhu, G Becherini, Y Peng, M Su… - arxiv e …, 2023 - ui.adsabs.harvard.edu
We propose EMAGE, a framework to generate full-body human gestures from audio and
masked gestures, encompassing facial, local body, hands, and global movements. To …

The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion

C Chen, J Zhang, SK Lakshmikanth, Y Fang… - arxiv preprint arxiv …, 2024 - arxiv.org
Human communication is inherently multimodal, involving a combination of verbal and non-
verbal cues such as speech, facial expressions, and body gestures. Modeling these …

MambaGesture: Enhancing Co-Speech Gesture Generation with Mamba and Disentangled Multi-Modality Fusion

C Fu, Y Wang, J Zhang, Z Jiang, X Mao, J Wu… - Proceedings of the …, 2024 - dl.acm.org
Co-speech gesture generation is crucial for producing synchronized and realistic human
gestures that accompany speech, enhancing the animation of lifelike avatars in virtual …