Abstract Video-to-audio (V2A) generation leverages visual-only video features to render
plausible sounds that match the scene. Importantly, the generated sound onsets should …

We introduce V-AURA, the first autoregressive model to achieve high temporal alignment
and relevance in video-to-audio generation. V-AURA uses a high-framerate visual feature …

Generating sound effects for videos often requires creating artistic sound effects that diverge
significantly from real-life sources and flexible control in the sound design. To address this …

We propose to synthesize high-quality and synchronized audio, given video and optional
text conditions, using a novel multimodal joint training framework MMAudio. In contrast to …

Video encompasses both visual and auditory data, creating a perceptually rich experience
where these two modalities complement each other. As such, videos are a valuable type of …

Video-to-audio (V2A) generation is important for video editing and post-processing,
enabling the creation of semantics-aligned audio for silent video. However, most existing …

Traditional Celluloid (Cel) Animation production pipeline encompasses multiple essential
steps, including storyboarding, layout design, keyframe animation, inbetweening, and …

Spectrograms are 2D representations of sound that look very different from the images found
in our visual world. And natural images, when played as spectrograms, make unnatural …

The interplay between audio and visual signals, rich in correlations across various scales,
significantly impacts human perception and drives a consistent demand for audio-visual …

Speech synthesis is a multifaceted process that encompasses both acoustic signals and
articulatory dynamics. Traditional neural audio synthesis methods often rely exclusively on …