Large motion model for unified multi-modal motion generation

M Zhang, D **, C Gu, F Hong, Z Cai, J Huang… - … on Computer Vision, 2024 - Springer
Human motion generation, a cornerstone technique in animation and video production, has
widespread applications in various tasks like text-to-motion and music-to-dance. Previous …

Semantic Gesticulator: Semantics-Aware Co-Speech Gesture Synthesis

Z Zhang, T Ao, Y Zhang, Q Gao, C Lin… - ACM Transactions on …, 2024 - dl.acm.org
In this work, we present Semantic Gesticulator, a novel framework designed to synthesize
realistic gestures accompanying speech with strong semantic correspondence. Semantically …

Lodge: A coarse to fine diffusion network for long dance generation guided by the characteristic dance primitives

R Li, YX Zhang, Y Zhang, H Zhang… - Proceedings of the …, 2024 - openaccess.thecvf.com
We propose Lodge a network capable of generating extremely long dance sequences
conditioned on given music. We design Lodge as a two-stage coarse to fine diffusion …

Mambatalk: Efficient holistic gesture synthesis with selective state space models

Z Xu, Y Lin, H Han, S Yang, R Li, Y Zhang… - arxiv preprint arxiv …, 2024 - arxiv.org
Gesture synthesis is a vital realm of human-computer interaction, with wide-ranging
applications across various fields like film, robotics, and virtual reality. Recent advancements …

Freetalker: Controllable speech and text-driven gesture generation based on diffusion models for enhanced speaker naturalness

S Yang, Z Xu, H Xue, Y Cheng, S Huang… - ICASSP 2024-2024 …, 2024 - ieeexplore.ieee.org
Current talking avatars mostly generate co-speech gestures based on audio and text of the
utterance, without considering the non-speaking motion of the speaker. Furthermore …

Synergistic Attention-Guided Cascaded Graph Diffusion Model for Complementarity Determining Region Synthesis

R Zhang, Y Huang, Y Lou, W Ding… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Complementarity determining region (CDR) is a specific region in antibody molecules that
binds to antigens, where a small portion of residues undergoes particularly pronounced …

Learning Co-Speech Gesture Representations in Dialogue through Contrastive Learning: An Intrinsic Evaluation

E Ghaleb, B Khaertdinov, W Pouw… - Proceedings of the 26th …, 2024 - dl.acm.org
In face-to-face dialogues, the form-meaning relationship of co-speech gestures varies
depending on contextual factors such as what the gestures refer to and the individual …

SemTalk: Holistic Co-speech Motion Generation with Frame-level Semantic Emphasis

X Zhang, J Li, J Zhang, Z Dang, J Ren, L Bo… - arxiv preprint arxiv …, 2024 - arxiv.org
A good co-speech motion generation cannot be achieved without a careful integration of
common rhythmic motion and rare yet essential semantic motion. In this work, we propose …

Alignment is All You Need: A Training-free Augmentation Strategy for Pose-guided Video Generation

X **, Z Xu, M Ou, W Yang - arxiv preprint arxiv:2408.16506, 2024 - arxiv.org
Character animation is a transformative field in computer graphics and vision, enabling
dynamic and realistic video animations from static images. Despite advancements …