Driven by recent advancements in Extended Reality (XR), the hype around the Metaverse,
and real‐time computer graphics, the transformation of the performing arts, particularly in …

We propose a self-supervised approach for learning to perform audio source separation in
videos based on natural language queries, using only unlabeled video and audio pairs as …

In music source separation, the number of sources may vary for each piece and some of the
sources may belong to the same family of instruments, thus sharing timbral characteristics …

Recent conditional and unconditional video generation tasks have been accomplished
mainly based on generative adversarial network (GAN), diffusion, and autoregressive …

We propose a novel system that takes as an input body movements of a musician playing a
musical instrument and generates music in an unsupervised setting. Learning to generate …

HuQin is a family of traditional Chinese bowed string instruments. Playing techniques (PTs)
embodied in various playing styles add abundant emotional coloring and aesthetic feelings …

In cross-modal music processing, translation between visual, auditory, and semantic content
opens up new possibilities as well as challenges. The construction of such a transformative …

For immersive applications, the generation of binaural sound that matches its visual
counterpart is crucial to bring meaningful experiences to people in a virtual environment …

Neural music instrument cloning is an application of deep neural networks for imitating the
timbre of a particular music instrument recording with a trained neural network. One can …

Recent MIDI-to-audio synthesis methods have employed deep neural networks to
successfully generate high-quality and expressive instrumental tracks. However, these …